2024-04-17T05:58:49.305 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T05:58:49.305 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi --archive /home/teuthworker/archive/jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi/7660092 --owner scheduled_jcollin@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/admin} -- /home/teuthworker/archive/jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi/7660092/orig.config.yaml 2024-04-17T05:58:49.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T05:58:49.438 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi/7660092 branch: wip-jcollin-testing_11042024 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/admin} email: jcollin@redhat.com first_in_suite: false job_id: '7660092' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-jcollin-testing_11042024 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: lockdep: true ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - missing required features - evicting unresponsive client sha1: 61531d579f13bd3cc7ef7522484ee03eec1daad6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 61531d579f13bd3cc7ef7522484ee03eec1daad6 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: origin-jcollin-testing sha1: 87915339f630682d277627759810c13965437320 owner: scheduled_jcollin@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8234 sha1: 61531d579f13bd3cc7ef7522484ee03eec1daad6 sleep_before_teardown: 0 suite: fs:functional suite_branch: origin-jcollin-testing suite_path: /home/teuthworker/src/github.com_joscollin_ceph_87915339f630682d277627759810c13965437320/qa suite_relpath: qa suite_repo: https://github.com/joscollin/ceph.git suite_sha1: 87915339f630682d277627759810c13965437320 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+TbV4fUXchRFWNVx30oQ2h5DoTR8mbuXmPYha/GkR4nWTQhahSzXGnBZw8o7dN0NHl1w0jVB7dwyzUAHp1Lqg= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIQsBI5FskDhZ/+GwSSt32xHi5zYHljFY8J7yzNAJXZ/UOIPksg1MKaRckUMeDFKeOwi/1ZOM8XDlbzMQppBSX8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-17_05:41:13 tube: smithi user: jcollin verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-17T05:58:49.438 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_joscollin_ceph_87915339f630682d277627759810c13965437320/qa; will attempt to use it 2024-04-17T05:58:49.439 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_joscollin_ceph_87915339f630682d277627759810c13965437320/qa/tasks 2024-04-17T05:58:49.439 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-17T05:58:49.440 INFO:teuthology.task.internal:Checking packages... 2024-04-17T05:58:49.464 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '61531d579f13bd3cc7ef7522484ee03eec1daad6' 2024-04-17T05:58:49.465 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-17T05:58:49.465 INFO:teuthology.packaging:ref: None 2024-04-17T05:58:49.465 INFO:teuthology.packaging:tag: None 2024-04-17T05:58:49.465 INFO:teuthology.packaging:branch: wip-jcollin-testing_11042024 2024-04-17T05:58:49.465 INFO:teuthology.packaging:sha1: 61531d579f13bd3cc7ef7522484ee03eec1daad6 2024-04-17T05:58:49.465 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-jcollin-testing_11042024 2024-04-17T05:58:49.679 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2862.g61531d57 2024-04-17T05:58:49.680 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-17T05:58:49.687 INFO:teuthology.task.internal:no buildpackages task found 2024-04-17T05:58:49.687 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-17T05:58:49.695 INFO:teuthology.task.internal:Saving configuration 2024-04-17T05:58:49.713 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-17T05:58:49.719 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-17T05:58:49.744 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi/7660092', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 05:53:41.260826', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+TbV4fUXchRFWNVx30oQ2h5DoTR8mbuXmPYha/GkR4nWTQhahSzXGnBZw8o7dN0NHl1w0jVB7dwyzUAHp1Lqg='} 2024-04-17T05:58:49.767 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi/7660092', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-17 05:53:41.262727', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIQsBI5FskDhZ/+GwSSt32xHi5zYHljFY8J7yzNAJXZ/UOIPksg1MKaRckUMeDFKeOwi/1ZOM8XDlbzMQppBSX8='} 2024-04-17T05:58:49.767 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-17T05:58:49.773 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-17T05:58:49.773 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-17T05:58:49.773 INFO:teuthology.run_tasks:Running task console_log... 2024-04-17T05:58:49.847 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0441f8e160>, signals=[15]) 2024-04-17T05:58:49.847 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-17T05:58:49.854 INFO:teuthology.task.internal:Opening connections... 2024-04-17T05:58:49.854 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T05:58:49.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:58:49.949 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T05:58:49.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T05:58:50.083 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-17T05:58:50.090 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-04-17T05:58:50.108 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-04-17T05:58:50.108 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-04-17T05:58:50.162 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-04-17T05:58:50.162 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T05:58:50.163 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T05:58:50.164 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-04-17T05:58:50.186 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2024-04-17T05:58:50.204 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2024-04-17T05:58:50.204 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2024-04-17T05:58:50.261 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2024-04-17T05:58:50.261 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2024-04-17T05:58:50.261 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2024-04-17T05:58:50.261 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2024-04-17T05:58:50.261 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2024-04-17T05:58:50.261 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2024-04-17T05:58:50.261 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T05:58:50.261 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2024-04-17T05:58:50.262 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2024-04-17T05:58:50.262 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T05:58:50.262 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2024-04-17T05:58:50.262 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T05:58:50.262 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T05:58:50.262 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T05:58:50.262 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2024-04-17T05:58:50.285 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-17T05:58:50.295 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-17T05:58:50.333 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-17T05:58:50.333 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-04-17T05:58:50.336 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2024-04-17T05:58:50.352 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-17T05:58:50.359 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-17T05:58:50.359 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-04-17T05:58:50.390 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2024-04-17T05:58:50.497 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-17T05:58:50.555 INFO:teuthology.run_tasks:Running task kernel... 2024-04-17T05:58:50.575 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-17T05:58:50.575 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-17T05:58:50.575 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-17T05:58:50.576 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T05:58:50.576 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-17T05:58:50.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:58:50.593 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-04-17T05:58:50.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T05:58:50.594 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-04-17T05:58:50.649 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-361.el9.x86_64 2024-04-17T05:58:50.649 INFO:teuthology.task.kernel:Running kernel on smithi135: 5.14.0-361.el9.x86_64 2024-04-17T05:58:50.649 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2024-04-17T05:58:50.650 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-361.el9.x86_64 2024-04-17T05:58:50.651 INFO:teuthology.task.kernel:Running kernel on smithi190: 5.14.0-361.el9.x86_64 2024-04-17T05:58:50.651 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum install -y kernel 2024-04-17T05:58:52.943 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-04-17T05:58:53.012 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.7 MB 00:00 2024-04-17T05:58:54.618 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-04-17T05:58:54.646 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-04-17T05:58:55.944 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-04-17T05:58:56.526 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-17T05:58:57.795 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-17T05:59:00.794 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-04-17T05:59:01.971 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-17T05:59:03.572 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-17T05:59:04.686 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-17T05:59:05.009 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-17T05:59:06.071 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 16 kB 00:00 2024-04-17T05:59:10.014 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-17T05:59:12.372 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-17T05:59:12.372 INFO:teuthology.orchestra.run.smithi190.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-17T05:59:12.452 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-04-17T05:59:12.460 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T05:59:12.460 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-04-17T05:59:12.460 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout:================================================================================ 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout:Install 4 Packages 2024-04-17T05:59:12.461 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T05:59:12.462 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 77 M 2024-04-17T05:59:12.462 INFO:teuthology.orchestra.run.smithi190.stdout:Installed size: 123 M 2024-04-17T05:59:12.462 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-04-17T05:59:12.608 INFO:teuthology.orchestra.run.smithi190.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.9 MB/s | 277 kB 00:00 2024-04-17T05:59:13.175 INFO:teuthology.orchestra.run.smithi190.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 21 MB/s | 15 MB 00:00 2024-04-17T05:59:13.791 INFO:teuthology.orchestra.run.smithi190.stdout:(3/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 25 MB/s | 34 MB 00:01 2024-04-17T05:59:14.058 INFO:teuthology.orchestra.run.smithi190.stdout:(4/4): kernel-modules-core-5.14.0-438.el9.x86_6 19 MB/s | 28 MB 00:01 2024-04-17T05:59:14.062 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-04-17T05:59:14.062 INFO:teuthology.orchestra.run.smithi190.stdout:Total 48 MB/s | 77 MB 00:01 2024-04-17T05:59:14.062 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-04-17T05:59:14.415 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-04-17T05:59:14.416 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-04-17T05:59:15.274 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-04-17T05:59:15.274 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-04-17T05:59:17.590 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-04-17T05:59:18.228 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-17T05:59:18.316 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T05:59:21.081 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T05:59:21.219 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T05:59:22.146 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 21 MB 00:15 2024-04-17T05:59:25.937 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T05:59:25.976 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:59:27.188 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-17T05:59:29.587 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-17T05:59:29.587 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-17T05:59:29.676 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-438.el9 CentOS-BaseOS 277 k 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core x86_64 5.14.0-438.el9 CentOS-BaseOS 15 M 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules x86_64 5.14.0-438.el9 CentOS-BaseOS 34 M 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core x86_64 5.14.0-438.el9 CentOS-BaseOS 28 M 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T05:59:29.684 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-17T05:59:29.685 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-04-17T05:59:29.685 INFO:teuthology.orchestra.run.smithi135.stdout:Install 4 Packages 2024-04-17T05:59:29.685 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T05:59:29.685 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 77 M 2024-04-17T05:59:29.685 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 123 M 2024-04-17T05:59:29.686 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-17T05:59:29.848 INFO:teuthology.orchestra.run.smithi135.stdout:(1/4): kernel-5.14.0-438.el9.x86_64.rpm 1.7 MB/s | 277 kB 00:00 2024-04-17T05:59:30.365 INFO:teuthology.orchestra.run.smithi135.stdout:(2/4): kernel-core-5.14.0-438.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-04-17T05:59:30.695 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:59:31.032 INFO:teuthology.orchestra.run.smithi135.stdout:(3/4): kernel-modules-core-5.14.0-438.el9.x86_6 23 MB/s | 28 MB 00:01 2024-04-17T05:59:31.382 INFO:teuthology.orchestra.run.smithi135.stdout:(4/4): kernel-modules-5.14.0-438.el9.x86_64.rpm 20 MB/s | 34 MB 00:01 2024-04-17T05:59:31.385 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-17T05:59:31.385 INFO:teuthology.orchestra.run.smithi135.stdout:Total 45 MB/s | 77 MB 00:01 2024-04-17T05:59:31.386 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-17T05:59:31.843 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-17T05:59:31.843 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-17T05:59:32.670 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-17T05:59:32.671 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-17T05:59:34.957 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-17T05:59:35.582 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-core-5.14.0-438.el9.x86_64 1/4 2024-04-17T05:59:35.687 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T05:59:38.440 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T05:59:38.590 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T05:59:43.307 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T05:59:43.423 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-5.14.0-438.el9.x86_64 4/4 2024-04-17T05:59:48.149 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:00.073 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:00.074 INFO:teuthology.orchestra.run.smithi190.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-17T06:00:00.074 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:00:00.117 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:00.452 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:00.452 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-17T06:00:00.452 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T06:00:00.452 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:00:00.870 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-04-17T06:00:01.196 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T06:00:01.297 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-17T06:00:01.297 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-17T06:00:01.297 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-17T06:00:01.297 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T06:00:01.297 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T06:00:01.298 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-04-17T06:00:01.298 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T06:00:01.298 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T06:00:01.378 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-17T06:00:01.378 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-17T06:00:01.379 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2024-04-17T06:00:01.421 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2024-04-17T06:00:01.421 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.06 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 68.el9 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Group : Unspecified 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 8264254 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : builder@centos.org 2024-04-17T06:00:01.422 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:00:01.423 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T06:00:01.424 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T06:00:01.424 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 5.14.0-438.el9.x86_64 2024-04-17T06:00:01.424 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T06:00:02.025 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2024-04-17T06:00:03.387 INFO:teuthology.orchestra.run.smithi190.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-17T06:00:03.422 INFO:teuthology.orchestra.run.smithi190.stderr:done 2024-04-17T06:00:03.424 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2024-04-17T06:00:03.439 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.8ckTelNpn3 2024-04-17T06:00:03.439 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8ckTelNpn3 2024-04-17T06:00:03.511 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.8ckTelNpn3 2024-04-17T06:00:03.651 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.8ckTelNpn3 is 6KB 2024-04-17T06:00:03.700 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.8ckTelNpn3 2024-04-17T06:00:03.715 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T06:00:03.778 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-17T06:00:03.779 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-17T06:00:03.779 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-17T06:00:03.779 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-17T06:00:03.780 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-17T06:00:03.933 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2024-04-17T06:00:17.622 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:17.623 INFO:teuthology.orchestra.run.smithi135.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-17T06:00:17.623 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:00:17.666 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-438.el9.x86_64 1/4 2024-04-17T06:00:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-core-5.14.0-438.el9.x86_64 2/4 2024-04-17T06:00:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-5.14.0-438.el9.x86_64 3/4 2024-04-17T06:00:18.540 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-core-5.14.0-438.el9.x86_64 4/4 2024-04-17T06:00:18.540 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:00:18.540 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-17T06:00:18.541 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-5.14.0-438.el9.x86_64 2024-04-17T06:00:18.541 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core-5.14.0-438.el9.x86_64 2024-04-17T06:00:18.541 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-5.14.0-438.el9.x86_64 2024-04-17T06:00:18.541 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core-5.14.0-438.el9.x86_64 2024-04-17T06:00:18.541 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:00:18.541 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-17T06:00:18.891 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T06:00:18.995 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-17T06:00:18.995 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-17T06:00:18.995 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-438.el9.x86_64 2024-04-17T06:00:18.995 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-17T06:00:18.995 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-17T06:00:18.995 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-17T06:00:18.996 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi135.front.sepia.ceph.com, path=None, version=distro) 2024-04-17T06:00:18.996 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-04-17T06:00:19.076 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-438.el9.x86_64 2024-04-17T06:00:19.077 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-438.el9.x86_64 2024-04-17T06:00:19.077 DEBUG:teuthology.orchestra.run.smithi135:> sudo rpm -qi grub2-tools 2024-04-17T06:00:19.111 INFO:teuthology.orchestra.run.smithi135.stdout:Name : grub2-tools 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Epoch : 1 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Version : 2.06 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Release : 68.el9 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Architecture: x86_64 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Group : Unspecified 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Size : 8264254 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:License : GPLv3+ 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Packager : builder@centos.org 2024-04-17T06:00:19.112 INFO:teuthology.orchestra.run.smithi135.stdout:Vendor : CentOS 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout:Summary : Support tools for GRUB. 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout:Description : 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout:hardware devices. 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:00:19.113 INFO:teuthology.orchestra.run.smithi135.stdout:This subpackage provides tools for support of all platforms. 2024-04-17T06:00:19.114 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-17T06:00:19.114 INFO:teuthology.task.kernel:Updating grub on smithi135 to boot 5.14.0-438.el9.x86_64 2024-04-17T06:00:19.114 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-17T06:00:19.764 INFO:teuthology.orchestra.run.smithi135.stderr:Generating grub configuration file ... 2024-04-17T06:00:21.126 INFO:teuthology.orchestra.run.smithi135.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-17T06:00:21.154 INFO:teuthology.orchestra.run.smithi135.stderr:done 2024-04-17T06:00:21.158 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2024-04-17T06:00:21.174 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.278TRrOUFp 2024-04-17T06:00:21.174 DEBUG:teuthology.orchestra.run.smithi135:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.278TRrOUFp 2024-04-17T06:00:21.242 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.278TRrOUFp 2024-04-17T06:00:21.382 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.278TRrOUFp is 6KB 2024-04-17T06:00:21.402 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.278TRrOUFp 2024-04-17T06:00:21.415 DEBUG:teuthology.orchestra.run.smithi135:> sudo /bin/ls /boot/loader/entries || true 2024-04-17T06:00:21.480 INFO:teuthology.orchestra.run.smithi135.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-17T06:00:21.480 INFO:teuthology.orchestra.run.smithi135.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-17T06:00:21.480 INFO:teuthology.orchestra.run.smithi135.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-17T06:00:21.480 INFO:teuthology.orchestra.run.smithi135.stdout:15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64.conf 2024-04-17T06:00:21.482 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-438.el9.x86_64 2024-04-17T06:00:21.655 DEBUG:teuthology.orchestra.run.smithi135:> sudo shutdown -r now 2024-04-17T06:00:33.942 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-17T06:00:33.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T06:00:33.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:00:51.674 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-17T06:00:51.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-17T06:00:51.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:00:52.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-04-17T06:01:01.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T06:01:01.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:01:04.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.190 2024-04-17T06:01:10.178 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.135 2024-04-17T06:01:16.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T06:01:16.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:01:19.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-17T06:01:19.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:01:22.242 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.135 2024-04-17T06:01:34.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-17T06:01:34.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:01:48.482 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-04-17T06:01:48.898 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T06:01:48.898 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-438.el9.x86_64"... 2024-04-17T06:01:48.898 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2024-04-17T06:01:48.913 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-438.el9.x86_64 2024-04-17T06:01:48.913 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-17T06:01:48.913 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T06:01:48.913 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T06:01:49.914 DEBUG:teuthology.parallel:result is None 2024-04-17T06:01:52.670 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.135 2024-04-17T06:02:07.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-17T06:02:07.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:02:08.032 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-04-17T06:02:08.495 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-17T06:02:08.495 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-438.el9.x86_64"... 2024-04-17T06:02:08.496 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-04-17T06:02:08.510 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-438.el9.x86_64 2024-04-17T06:02:08.510 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-438.el9.x86_64 vs 5.14.0-438.el9.x86_64 2024-04-17T06:02:08.510 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-17T06:02:08.510 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-17T06:02:09.511 DEBUG:teuthology.parallel:result is None 2024-04-17T06:02:09.512 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-17T06:02:09.519 INFO:teuthology.task.internal:Creating test directory... 2024-04-17T06:02:09.519 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T06:02:09.521 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-17T06:02:09.544 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-17T06:02:09.550 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-17T06:02:09.560 INFO:teuthology.task.internal:Creating archive directory... 2024-04-17T06:02:09.561 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T06:02:09.583 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-17T06:02:09.625 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-17T06:02:09.632 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-17T06:02:09.632 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T06:02:09.658 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-17T06:02:09.751 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:02:09.763 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:02:09.794 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:02:09.822 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-17T06:02:09.825 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-17T06:02:09.832 INFO:teuthology.task.internal:Configuring sudo... 2024-04-17T06:02:09.832 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T06:02:09.836 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-17T06:02:09.880 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-17T06:02:09.889 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-17T06:02:09.889 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T06:02:09.892 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-17T06:02:09.925 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:02:10.008 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:02:10.078 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:02:10.126 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:02:10.171 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:02:10.171 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T06:02:10.239 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:02:10.280 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-17T06:02:10.335 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:02:10.418 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-17T06:02:10.463 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:02:10.463 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-17T06:02:10.533 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-04-17T06:02:10.536 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2024-04-17T06:02:10.640 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T06:02:10.663 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-17T06:02:11.139 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-17T06:02:11.147 INFO:teuthology.task.internal:Starting timer... 2024-04-17T06:02:11.147 INFO:teuthology.run_tasks:Running task pcp... 2024-04-17T06:02:11.160 INFO:teuthology.run_tasks:Running task selinux... 2024-04-17T06:02:11.169 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-17T06:02:11.169 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-04-17T06:02:11.240 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-04-17T06:02:11.242 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2024-04-17T06:02:11.307 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2024-04-17T06:02:11.309 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-17T06:02:11.309 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-04-17T06:02:11.334 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-04-17T06:02:11.334 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2024-04-17T06:02:11.359 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2024-04-17T06:02:11.359 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi135.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2024-04-17T06:02:11.359 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T06:02:11.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:02:11.401 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-17T06:02:11.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:02:11.429 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-17T06:02:11.430 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-04-17T06:02:11.478 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2024-04-17T06:02:11.511 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-17T06:02:11.520 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-17T06:02:11.525 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-17T06:02:11.661 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-17T06:02:11.677 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-17T06:02:11.678 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi135.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-17T06:06:24.849 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi135.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2024-04-17T06:06:24.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-17T06:06:24.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:06:24.944 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-04-17T06:06:25.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-04-17T06:06:25.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T06:06:25.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T06:06:25.115 DEBUG:teuthology.orchestra.run.smithi190:> true 2024-04-17T06:06:25.203 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2024-04-17T06:06:25.203 INFO:teuthology.run_tasks:Running task clock... 2024-04-17T06:06:25.216 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-17T06:06:25.217 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T06:06:25.217 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T06:06:25.220 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-17T06:06:25.220 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-17T06:06:25.250 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T06:06:25.267 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T06:06:25.277 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-17T06:06:25.295 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-17T06:06:25.298 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-04-17T06:06:25.313 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-04-17T06:06:25.326 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2024-04-17T06:06:25.330 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T06:06:25.341 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2024-04-17T06:06:25.347 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T06:06:25.358 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-17T06:06:25.376 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-17T06:06:25.395 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-04-17T06:06:25.399 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T06:06:25.399 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-04-17T06:06:25.399 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:06:25.400 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:06:25.400 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:06:25.400 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:06:25.431 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2024-04-17T06:06:25.434 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-17T06:06:25.435 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2024-04-17T06:06:25.435 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:06:25.435 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:06:25.435 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:06:25.435 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-17T06:06:25.436 INFO:teuthology.run_tasks:Running task install... 2024-04-17T06:06:25.444 DEBUG:teuthology.task.install:project ceph 2024-04-17T06:06:25.444 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6'}} 2024-04-17T06:06:25.445 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6'} 2024-04-17T06:06:25.445 INFO:teuthology.task.install:Using flavor: default 2024-04-17T06:06:25.450 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-17T06:06:25.451 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-17T06:06:25.451 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6', 'tag': None, 'wait_for_package': False} 2024-04-17T06:06:25.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=61531d579f13bd3cc7ef7522484ee03eec1daad6 2024-04-17T06:06:25.453 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6', 'tag': None, 'wait_for_package': False} 2024-04-17T06:06:25.454 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=61531d579f13bd3cc7ef7522484ee03eec1daad6 2024-04-17T06:06:25.598 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing_11042024/61531d579f13bd3cc7ef7522484ee03eec1daad6/centos/9/flavors/default/ 2024-04-17T06:06:25.599 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2862.g61531d57 2024-04-17T06:06:25.669 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing_11042024/61531d579f13bd3cc7ef7522484ee03eec1daad6/centos/9/flavors/default/ 2024-04-17T06:06:25.669 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2862.g61531d57 2024-04-17T06:06:25.769 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing_11042024/61531d579f13bd3cc7ef7522484ee03eec1daad6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing_11042024/61531d579f13bd3cc7ef7522484ee03eec1daad6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing_11042024/61531d579f13bd3cc7ef7522484ee03eec1daad6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T06:06:25.769 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:06:25.769 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T06:06:25.799 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T06:06:25.799 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/61531d579f13bd3cc7ef7522484ee03eec1daad6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T06:06:25.867 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T06:06:25.949 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T06:06:26.001 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing_11042024/61531d579f13bd3cc7ef7522484ee03eec1daad6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing_11042024/61531d579f13bd3cc7ef7522484ee03eec1daad6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing_11042024/61531d579f13bd3cc7ef7522484ee03eec1daad6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-17T06:06:26.001 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:06:26.001 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-17T06:06:26.017 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2024-04-17T06:06:26.019 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2024-04-17T06:06:26.033 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-17T06:06:26.033 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/61531d579f13bd3cc7ef7522484ee03eec1daad6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-17T06:06:26.105 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-17T06:06:26.189 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-17T06:06:26.258 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-04-17T06:06:26.260 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-04-17T06:06:26.263 INFO:teuthology.orchestra.run.smithi190.stdout:74 files removed 2024-04-17T06:06:26.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T06:06:26.507 INFO:teuthology.orchestra.run.smithi135.stdout:74 files removed 2024-04-17T06:06:26.533 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-17T06:06:27.101 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 165 kB/s | 83 kB 00:00 2024-04-17T06:06:27.216 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 221 kB/s | 83 kB 00:00 2024-04-17T06:06:27.625 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-17T06:06:27.777 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-04-17T06:06:28.043 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-17T06:06:28.402 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-04-17T06:06:28.753 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-04-17T06:06:28.822 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-04-17T06:06:30.222 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-04-17T06:06:30.387 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-04-17T06:06:30.979 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-17T06:06:31.169 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-04-17T06:06:33.179 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-17T06:06:34.605 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-04-17T06:06:37.271 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-17T06:06:38.789 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.2 MB 00:00 2024-04-17T06:06:38.797 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-17T06:06:40.265 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-17T06:06:42.048 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 21 MB 00:03 2024-04-17T06:06:47.098 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-04-17T06:06:49.834 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:06:49.835 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:06:49.847 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T06:06:49.848 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T06:06:49.849 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T06:06:49.850 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T06:06:49.851 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T06:06:49.854 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T06:06:49.855 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T06:06:49.938 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-04-17T06:06:49.955 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2024-04-17T06:06:49.955 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-04-17T06:06:49.955 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:19.0.0-2862.g61531d57.el9 ceph 6.6 k 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:19.0.0-2862.g61531d57.el9 ceph 5.4 M 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:19.0.0-2862.g61531d57.el9 ceph 891 k 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2862.g61531d57.el9 ceph 145 k 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:19.0.0-2862.g61531d57.el9 ceph 1.5 M 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 142 k 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 3.5 M 2024-04-17T06:06:49.956 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 7.4 M 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 49 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:19.0.0-2862.g61531d57.el9 ceph 10 M 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:19.0.0-2862.g61531d57.el9 ceph 49 M 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 264 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 762 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:19.0.0-2862.g61531d57.el9 ceph 225 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell x86_64 2:19.0.0-2862.g61531d57.el9 ceph 27 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 18 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:19.0.0-2862.g61531d57.el9 ceph 32 k 2024-04-17T06:06:49.957 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 746 k 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:19.0.0-2862.g61531d57.el9 ceph 127 k 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:19.0.0-2862.g61531d57.el9 ceph 161 k 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:19.0.0-2862.g61531d57.el9 ceph 321 k 2024-04-17T06:06:49.958 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:19.0.0-2862.g61531d57.el9 ceph 297 k 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:19.0.0-2862.g61531d57.el9 ceph 99 k 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:19.0.0-2862.g61531d57.el9 ceph 85 k 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:19.0.0-2862.g61531d57.el9 ceph 3.1 M 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:19.0.0-2862.g61531d57.el9 ceph 170 k 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T06:06:49.959 INFO:teuthology.orchestra.run.smithi135.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 3.4 M 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 3.1 M 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T06:06:49.960 INFO:teuthology.orchestra.run.smithi135.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:19.0.0-2862.g61531d57.el9 ceph 21 M 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 34 k 2024-04-17T06:06:49.961 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:19.0.0-2862.g61531d57.el9 ceph 2.4 M 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 249 k 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:19.0.0-2862.g61531d57.el9 ceph 4.7 M 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:19.0.0-2862.g61531d57.el9 ceph 17 M 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 16 k 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:19.0.0-2862.g61531d57.el9 ceph 25 k 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T06:06:49.962 INFO:teuthology.orchestra.run.smithi135.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:19.0.0-2862.g61531d57.el9 ceph 163 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 501 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 5.2 M 2024-04-17T06:06:49.963 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T06:06:49.964 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:19.0.0-2862.g61531d57.el9 ceph 45 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:19.0.0-2862.g61531d57.el9 ceph 135 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T06:06:49.965 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T06:06:49.966 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T06:06:49.967 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T06:06:49.968 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T06:06:49.969 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-04-17T06:06:49.970 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T06:06:49.970 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:06:49.970 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-04-17T06:06:49.970 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2024-04-17T06:06:49.970 INFO:teuthology.orchestra.run.smithi135.stdout:Install 150 Packages 2024-04-17T06:06:49.970 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 24 Packages 2024-04-17T06:06:49.970 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:06:49.970 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 228 M 2024-04-17T06:06:49.971 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-04-17T06:06:50.615 INFO:teuthology.orchestra.run.smithi135.stdout:(1/174): ceph-19.0.0-2862.g61531d57.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-04-17T06:06:50.841 INFO:teuthology.orchestra.run.smithi135.stdout:(2/174): ceph-fuse-19.0.0-2862.g61531d57.el9.x8 3.9 MB/s | 891 kB 00:00 2024-04-17T06:06:50.907 INFO:teuthology.orchestra.run.smithi135.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 2.1 MB/s | 145 kB 00:00 2024-04-17T06:06:51.241 INFO:teuthology.orchestra.run.smithi135.stdout:(4/174): ceph-mds-19.0.0-2862.g61531d57.el9.x86 7.1 MB/s | 2.4 MB 00:00 2024-04-17T06:06:51.425 INFO:teuthology.orchestra.run.smithi135.stdout:(5/174): ceph-mgr-19.0.0-2862.g61531d57.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-04-17T06:06:51.550 INFO:teuthology.orchestra.run.smithi135.stdout:(6/174): ceph-base-19.0.0-2862.g61531d57.el9.x8 4.9 MB/s | 5.4 MB 00:01 2024-04-17T06:06:51.984 INFO:teuthology.orchestra.run.smithi135.stdout:(7/174): ceph-mon-19.0.0-2862.g61531d57.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-04-17T06:06:52.376 INFO:teuthology.orchestra.run.smithi135.stdout:(8/174): ceph-common-19.0.0-2862.g61531d57.el9. 11 MB/s | 21 MB 00:01 2024-04-17T06:06:52.443 INFO:teuthology.orchestra.run.smithi135.stdout:(9/174): ceph-selinux-19.0.0-2862.g61531d57.el9 379 kB/s | 25 kB 00:00 2024-04-17T06:06:53.018 INFO:teuthology.orchestra.run.smithi135.stdout:(10/174): ceph-radosgw-19.0.0-2862.g61531d57.el 9.6 MB/s | 10 MB 00:01 2024-04-17T06:06:53.101 INFO:teuthology.orchestra.run.smithi135.stdout:(11/174): cephfs-mirror-19.0.0-2862.g61531d57.e 2.6 MB/s | 225 kB 00:00 2024-04-17T06:06:53.177 INFO:teuthology.orchestra.run.smithi135.stdout:(12/174): cephfs-shell-19.0.0-2862.g61531d57.el 360 kB/s | 27 kB 00:00 2024-04-17T06:06:53.252 INFO:teuthology.orchestra.run.smithi135.stdout:(13/174): libcephfs-devel-19.0.0-2862.g61531d57 423 kB/s | 32 kB 00:00 2024-04-17T06:06:53.419 INFO:teuthology.orchestra.run.smithi135.stdout:(14/174): libcephfs2-19.0.0-2862.g61531d57.el9. 4.4 MB/s | 746 kB 00:00 2024-04-17T06:06:53.486 INFO:teuthology.orchestra.run.smithi135.stdout:(15/174): libcephsqlite-19.0.0-2862.g61531d57.e 2.4 MB/s | 163 kB 00:00 2024-04-17T06:06:53.552 INFO:teuthology.orchestra.run.smithi135.stdout:(16/174): librados-devel-19.0.0-2862.g61531d57. 1.9 MB/s | 127 kB 00:00 2024-04-17T06:06:53.636 INFO:teuthology.orchestra.run.smithi135.stdout:(17/174): libradosstriper1-19.0.0-2862.g61531d5 5.9 MB/s | 501 kB 00:00 2024-04-17T06:06:53.828 INFO:teuthology.orchestra.run.smithi135.stdout:(18/174): ceph-osd-19.0.0-2862.g61531d57.el9.x8 7.4 MB/s | 17 MB 00:02 2024-04-17T06:06:53.920 INFO:teuthology.orchestra.run.smithi135.stdout:(19/174): python3-ceph-argparse-19.0.0-2862.g61 494 kB/s | 45 kB 00:00 2024-04-17T06:06:54.012 INFO:teuthology.orchestra.run.smithi135.stdout:(20/174): python3-ceph-common-19.0.0-2862.g6153 1.4 MB/s | 135 kB 00:00 2024-04-17T06:06:54.146 INFO:teuthology.orchestra.run.smithi135.stdout:(21/174): librgw2-19.0.0-2862.g61531d57.el9.x86 10 MB/s | 5.2 MB 00:00 2024-04-17T06:06:54.170 INFO:teuthology.orchestra.run.smithi135.stdout:(22/174): python3-cephfs-19.0.0-2862.g61531d57. 1.0 MB/s | 161 kB 00:00 2024-04-17T06:06:54.196 INFO:teuthology.orchestra.run.smithi135.stdout:(23/174): python3-rados-19.0.0-2862.g61531d57.e 6.3 MB/s | 321 kB 00:00 2024-04-17T06:06:54.238 INFO:teuthology.orchestra.run.smithi135.stdout:(24/174): python3-rbd-19.0.0-2862.g61531d57.el9 4.3 MB/s | 297 kB 00:00 2024-04-17T06:06:54.263 INFO:teuthology.orchestra.run.smithi135.stdout:(25/174): python3-rgw-19.0.0-2862.g61531d57.el9 1.4 MB/s | 99 kB 00:00 2024-04-17T06:06:54.305 INFO:teuthology.orchestra.run.smithi135.stdout:(26/174): rbd-fuse-19.0.0-2862.g61531d57.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-17T06:06:54.555 INFO:teuthology.orchestra.run.smithi135.stdout:(27/174): rbd-mirror-19.0.0-2862.g61531d57.el9. 11 MB/s | 3.1 MB 00:00 2024-04-17T06:06:54.605 INFO:teuthology.orchestra.run.smithi135.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2862.g 685 kB/s | 34 kB 00:00 2024-04-17T06:06:54.631 INFO:teuthology.orchestra.run.smithi135.stdout:(29/174): rbd-nbd-19.0.0-2862.g61531d57.el9.x86 523 kB/s | 170 kB 00:00 2024-04-17T06:06:54.664 INFO:teuthology.orchestra.run.smithi135.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2862.g61531d5 2.4 MB/s | 142 kB 00:00 2024-04-17T06:06:55.240 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 1.4 MB/s | 21 MB 00:14 2024-04-17T06:06:55.290 INFO:teuthology.orchestra.run.smithi135.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2862.g61531 5.4 MB/s | 3.5 MB 00:00 2024-04-17T06:06:55.420 INFO:teuthology.orchestra.run.smithi135.stdout:(32/174): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-17T06:06:55.432 INFO:teuthology.orchestra.run.smithi135.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2862.g61 1.7 MB/s | 249 kB 00:00 2024-04-17T06:06:55.457 INFO:teuthology.orchestra.run.smithi135.stdout:(34/174): ceph-mgr-rook-19.0.0-2862.g61531d57.e 846 kB/s | 49 kB 00:00 2024-04-17T06:06:55.499 INFO:teuthology.orchestra.run.smithi135.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2862.g6 246 kB/s | 16 kB 00:00 2024-04-17T06:06:55.524 INFO:teuthology.orchestra.run.smithi135.stdout:(36/174): ceph-volume-19.0.0-2862.g61531d57.el9 3.8 MB/s | 264 kB 00:00 2024-04-17T06:06:55.591 INFO:teuthology.orchestra.run.smithi135.stdout:(37/174): cephfs-top-19.0.0-2862.g61531d57.el9. 274 kB/s | 18 kB 00:00 2024-04-17T06:06:55.659 INFO:teuthology.orchestra.run.smithi135.stdout:(38/174): cephadm-19.0.0-2862.g61531d57.el9.noa 4.7 MB/s | 762 kB 00:00 2024-04-17T06:06:56.147 INFO:teuthology.orchestra.run.smithi135.stdout:(39/174): ceph-test-19.0.0-2862.g61531d57.el9.x 13 MB/s | 49 MB 00:03 2024-04-17T06:06:56.334 INFO:teuthology.orchestra.run.smithi135.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 920 kB 00:00 2024-04-17T06:06:56.359 INFO:teuthology.orchestra.run.smithi135.stdout:(41/174): automake-1.16.2-8.el9.noarch.rpm 862 kB/s | 662 kB 00:00 2024-04-17T06:06:56.384 INFO:teuthology.orchestra.run.smithi135.stdout:(42/174): boost-program-options-1.75.0-8.el9.x8 446 kB/s | 106 kB 00:00 2024-04-17T06:06:56.409 INFO:teuthology.orchestra.run.smithi135.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-04-17T06:06:56.426 INFO:teuthology.orchestra.run.smithi135.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 769 kB/s | 32 kB 00:00 2024-04-17T06:06:56.577 INFO:teuthology.orchestra.run.smithi135.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-17T06:06:56.593 INFO:teuthology.orchestra.run.smithi135.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 93 kB/s | 17 kB 00:00 2024-04-17T06:06:56.610 INFO:teuthology.orchestra.run.smithi135.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 55 kB/s | 10 kB 00:00 2024-04-17T06:06:56.644 INFO:teuthology.orchestra.run.smithi135.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-17T06:06:56.669 INFO:teuthology.orchestra.run.smithi135.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-04-17T06:06:56.703 INFO:teuthology.orchestra.run.smithi135.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2024-04-17T06:06:56.728 INFO:teuthology.orchestra.run.smithi135.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-04-17T06:06:56.745 INFO:teuthology.orchestra.run.smithi135.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 622 kB/s | 46 kB 00:00 2024-04-17T06:06:56.770 INFO:teuthology.orchestra.run.smithi135.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-04-17T06:06:56.804 INFO:teuthology.orchestra.run.smithi135.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 187 kB 00:00 2024-04-17T06:06:56.829 INFO:teuthology.orchestra.run.smithi135.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.5 MB/s | 217 kB 00:00 2024-04-17T06:06:56.846 INFO:teuthology.orchestra.run.smithi135.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 454 kB/s | 34 kB 00:00 2024-04-17T06:06:56.988 INFO:teuthology.orchestra.run.smithi135.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-17T06:06:57.022 INFO:teuthology.orchestra.run.smithi135.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 817 kB 00:00 2024-04-17T06:06:57.063 INFO:teuthology.orchestra.run.smithi135.stdout:(59/174): postgresql-13.14-1.el9.x86_64.rpm 7.1 MB/s | 1.5 MB 00:00 2024-04-17T06:06:57.088 INFO:teuthology.orchestra.run.smithi135.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 132 kB 00:00 2024-04-17T06:06:57.144 INFO:teuthology.orchestra.run.smithi135.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.4 MB/s | 78 kB 00:00 2024-04-17T06:06:57.222 INFO:teuthology.orchestra.run.smithi135.stdout:(62/174): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.7 MB 00:00 2024-04-17T06:06:57.256 INFO:teuthology.orchestra.run.smithi135.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-17T06:06:57.356 INFO:teuthology.orchestra.run.smithi135.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-04-17T06:06:57.390 INFO:teuthology.orchestra.run.smithi135.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 186 kB/s | 24 kB 00:00 2024-04-17T06:06:57.415 INFO:teuthology.orchestra.run.smithi135.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-04-17T06:06:57.440 INFO:teuthology.orchestra.run.smithi135.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-04-17T06:06:57.465 INFO:teuthology.orchestra.run.smithi135.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 636 kB/s | 32 kB 00:00 2024-04-17T06:06:57.566 INFO:teuthology.orchestra.run.smithi135.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-04-17T06:06:57.591 INFO:teuthology.orchestra.run.smithi135.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-04-17T06:06:57.616 INFO:teuthology.orchestra.run.smithi135.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-04-17T06:06:57.641 INFO:teuthology.orchestra.run.smithi135.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 921 kB/s | 69 kB 00:00 2024-04-17T06:06:57.667 INFO:teuthology.orchestra.run.smithi135.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 819 kB/s | 41 kB 00:00 2024-04-17T06:06:57.692 INFO:teuthology.orchestra.run.smithi135.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-17T06:06:57.717 INFO:teuthology.orchestra.run.smithi135.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-17T06:06:57.743 INFO:teuthology.orchestra.run.smithi135.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-04-17T06:06:57.768 INFO:teuthology.orchestra.run.smithi135.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-17T06:06:57.784 INFO:teuthology.orchestra.run.smithi135.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 43 kB 00:00 2024-04-17T06:06:57.927 INFO:teuthology.orchestra.run.smithi135.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 295 kB/s | 41 kB 00:00 2024-04-17T06:06:57.943 INFO:teuthology.orchestra.run.smithi135.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 247 kB/s | 39 kB 00:00 2024-04-17T06:06:58.119 INFO:teuthology.orchestra.run.smithi135.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-04-17T06:06:58.152 INFO:teuthology.orchestra.run.smithi135.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 975 kB/s | 220 kB 00:00 2024-04-17T06:06:58.169 INFO:teuthology.orchestra.run.smithi135.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 156 kB/s | 35 kB 00:00 2024-04-17T06:06:58.186 INFO:teuthology.orchestra.run.smithi135.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 700 kB/s | 46 kB 00:00 2024-04-17T06:06:58.203 INFO:teuthology.orchestra.run.smithi135.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2024-04-17T06:06:58.228 INFO:teuthology.orchestra.run.smithi135.stdout:(86/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-04-17T06:06:58.253 INFO:teuthology.orchestra.run.smithi135.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 932 kB/s | 63 kB 00:00 2024-04-17T06:06:58.279 INFO:teuthology.orchestra.run.smithi135.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-04-17T06:06:58.304 INFO:teuthology.orchestra.run.smithi135.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-17T06:06:58.329 INFO:teuthology.orchestra.run.smithi135.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-17T06:06:58.363 INFO:teuthology.orchestra.run.smithi135.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-17T06:06:58.388 INFO:teuthology.orchestra.run.smithi135.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-04-17T06:06:58.413 INFO:teuthology.orchestra.run.smithi135.stdout:(93/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-04-17T06:06:58.438 INFO:teuthology.orchestra.run.smithi135.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-04-17T06:06:58.580 INFO:teuthology.orchestra.run.smithi135.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 977 kB/s | 187 kB 00:00 2024-04-17T06:06:58.605 INFO:teuthology.orchestra.run.smithi135.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-17T06:06:58.622 INFO:teuthology.orchestra.run.smithi135.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-17T06:06:58.656 INFO:teuthology.orchestra.run.smithi135.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-17T06:06:58.681 INFO:teuthology.orchestra.run.smithi135.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-04-17T06:06:58.715 INFO:teuthology.orchestra.run.smithi135.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 276 kB/s | 25 kB 00:00 2024-04-17T06:06:58.781 INFO:teuthology.orchestra.run.smithi135.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.4 MB/s | 245 kB 00:00 2024-04-17T06:06:58.807 INFO:teuthology.orchestra.run.smithi135.stdout:(102/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.6 MB/s | 248 kB 00:00 2024-04-17T06:06:58.832 INFO:teuthology.orchestra.run.smithi135.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-17T06:06:58.857 INFO:teuthology.orchestra.run.smithi135.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-04-17T06:06:58.966 INFO:teuthology.orchestra.run.smithi135.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 13 MB/s | 4.1 MB 00:00 2024-04-17T06:06:59.000 INFO:teuthology.orchestra.run.smithi135.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-04-17T06:06:59.170 INFO:teuthology.orchestra.run.smithi135.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 138 kB/s | 19 kB 00:00 2024-04-17T06:06:59.217 INFO:teuthology.orchestra.run.smithi135.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-17T06:06:59.242 INFO:teuthology.orchestra.run.smithi135.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-04-17T06:06:59.267 INFO:teuthology.orchestra.run.smithi135.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-04-17T06:06:59.293 INFO:teuthology.orchestra.run.smithi135.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-04-17T06:06:59.318 INFO:teuthology.orchestra.run.smithi135.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-17T06:06:59.352 INFO:teuthology.orchestra.run.smithi135.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-17T06:06:59.502 INFO:teuthology.orchestra.run.smithi135.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-17T06:06:59.544 INFO:teuthology.orchestra.run.smithi135.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-04-17T06:06:59.561 INFO:teuthology.orchestra.run.smithi135.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-04-17T06:06:59.578 INFO:teuthology.orchestra.run.smithi135.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 961 kB/s | 32 kB 00:00 2024-04-17T06:06:59.594 INFO:teuthology.orchestra.run.smithi135.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 445 kB/s | 15 kB 00:00 2024-04-17T06:06:59.636 INFO:teuthology.orchestra.run.smithi135.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-17T06:06:59.712 INFO:teuthology.orchestra.run.smithi135.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-17T06:06:59.737 INFO:teuthology.orchestra.run.smithi135.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-17T06:06:59.762 INFO:teuthology.orchestra.run.smithi135.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 702 kB/s | 35 kB 00:00 2024-04-17T06:06:59.821 INFO:teuthology.orchestra.run.smithi135.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-17T06:06:59.879 INFO:teuthology.orchestra.run.smithi135.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-17T06:06:59.922 INFO:teuthology.orchestra.run.smithi135.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-17T06:06:59.963 INFO:teuthology.orchestra.run.smithi135.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 426 kB/s | 18 kB 00:00 2024-04-17T06:07:00.005 INFO:teuthology.orchestra.run.smithi135.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 557 kB/s | 23 kB 00:00 2024-04-17T06:07:00.047 INFO:teuthology.orchestra.run.smithi135.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 469 kB/s | 19 kB 00:00 2024-04-17T06:07:00.089 INFO:teuthology.orchestra.run.smithi135.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 471 kB/s | 20 kB 00:00 2024-04-17T06:07:00.239 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-17T06:07:00.290 INFO:teuthology.orchestra.run.smithi135.stdout:(130/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-04-17T06:07:00.331 INFO:teuthology.orchestra.run.smithi135.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-17T06:07:00.398 INFO:teuthology.orchestra.run.smithi135.stdout:(132/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-04-17T06:07:00.440 INFO:teuthology.orchestra.run.smithi135.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-04-17T06:07:00.465 INFO:teuthology.orchestra.run.smithi135.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 589 kB/s | 79 kB 00:00 2024-04-17T06:07:00.491 INFO:teuthology.orchestra.run.smithi135.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-04-17T06:07:00.516 INFO:teuthology.orchestra.run.smithi135.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-04-17T06:07:00.541 INFO:teuthology.orchestra.run.smithi135.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-17T06:07:00.566 INFO:teuthology.orchestra.run.smithi135.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-04-17T06:07:00.592 INFO:teuthology.orchestra.run.smithi135.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-04-17T06:07:00.618 INFO:teuthology.orchestra.run.smithi135.stdout:(140/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-17T06:07:00.659 INFO:teuthology.orchestra.run.smithi135.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 887 kB/s | 59 kB 00:00 2024-04-17T06:07:00.684 INFO:teuthology.orchestra.run.smithi135.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-04-17T06:07:00.710 INFO:teuthology.orchestra.run.smithi135.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-17T06:07:00.760 INFO:teuthology.orchestra.run.smithi135.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 681 kB/s | 51 kB 00:00 2024-04-17T06:07:00.793 INFO:teuthology.orchestra.run.smithi135.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-04-17T06:07:00.835 INFO:teuthology.orchestra.run.smithi135.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-04-17T06:07:00.877 INFO:teuthology.orchestra.run.smithi135.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-17T06:07:00.902 INFO:teuthology.orchestra.run.smithi135.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-17T06:07:00.970 INFO:teuthology.orchestra.run.smithi135.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-17T06:07:01.278 INFO:teuthology.orchestra.run.smithi135.stdout:(150/174): librbd1-19.0.0-2862.g61531d57.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-17T06:07:01.362 INFO:teuthology.orchestra.run.smithi135.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 28 MB/s | 2.3 MB 00:00 2024-04-17T06:07:01.462 INFO:teuthology.orchestra.run.smithi135.stdout:(152/174): librados2-19.0.0-2862.g61531d57.el9. 6.1 MB/s | 3.4 MB 00:00 2024-04-17T06:07:01.504 INFO:teuthology.orchestra.run.smithi135.stdout:(153/174): NetworkManager-libnm-1.47.4-1.el9.x8 13 MB/s | 1.8 MB 00:00 2024-04-17T06:07:01.529 INFO:teuthology.orchestra.run.smithi135.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 612 kB/s | 41 kB 00:00 2024-04-17T06:07:01.554 INFO:teuthology.orchestra.run.smithi135.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.8 MB/s | 248 kB 00:00 2024-04-17T06:07:01.580 INFO:teuthology.orchestra.run.smithi135.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-17T06:07:01.647 INFO:teuthology.orchestra.run.smithi135.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 783 kB/s | 71 kB 00:00 2024-04-17T06:07:01.672 INFO:teuthology.orchestra.run.smithi135.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 397 kB/s | 36 kB 00:00 2024-04-17T06:07:01.722 INFO:teuthology.orchestra.run.smithi135.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-04-17T06:07:01.764 INFO:teuthology.orchestra.run.smithi135.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-04-17T06:07:01.806 INFO:teuthology.orchestra.run.smithi135.stdout:(161/174): thrift-0.15.0-2.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-04-17T06:07:01.823 INFO:teuthology.orchestra.run.smithi135.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-04-17T06:07:01.848 INFO:teuthology.orchestra.run.smithi135.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 108 kB 00:00 2024-04-17T06:07:01.873 INFO:teuthology.orchestra.run.smithi135.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-17T06:07:01.898 INFO:teuthology.orchestra.run.smithi135.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-17T06:07:01.924 INFO:teuthology.orchestra.run.smithi135.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 572 kB/s | 29 kB 00:00 2024-04-17T06:07:01.949 INFO:teuthology.orchestra.run.smithi135.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-04-17T06:07:02.008 INFO:teuthology.orchestra.run.smithi135.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-04-17T06:07:02.066 INFO:teuthology.orchestra.run.smithi135.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.8 MB/s | 465 kB 00:00 2024-04-17T06:07:02.117 INFO:teuthology.orchestra.run.smithi135.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-04-17T06:07:02.150 INFO:teuthology.orchestra.run.smithi135.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 371 kB/s | 31 kB 00:00 2024-04-17T06:07:02.201 INFO:teuthology.orchestra.run.smithi135.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.1 MB/s | 2.3 MB 00:00 2024-04-17T06:07:02.234 INFO:teuthology.orchestra.run.smithi135.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.2 MB/s | 1.1 MB 00:00 2024-04-17T06:07:02.331 INFO:teuthology.orchestra.run.smithi135.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 413 kB/s | 74 kB 00:00 2024-04-17T06:07:02.338 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-04-17T06:07:02.338 INFO:teuthology.orchestra.run.smithi135.stdout:Total 18 MB/s | 228 MB 00:12 2024-04-17T06:07:02.737 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-04-17T06:07:02.927 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:07:02.928 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-17T06:07:02.942 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-17T06:07:02.943 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-17T06:07:02.944 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-17T06:07:02.945 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-17T06:07:02.947 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-17T06:07:02.950 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-17T06:07:02.951 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-17T06:07:03.011 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-04-17T06:07:03.011 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-04-17T06:07:03.031 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2024-04-17T06:07:03.049 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:19.0.0-2862.g61531d57.el9 ceph 6.6 k 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:19.0.0-2862.g61531d57.el9 ceph 5.4 M 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:19.0.0-2862.g61531d57.el9 ceph 891 k 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2862.g61531d57.el9 ceph 145 k 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:19.0.0-2862.g61531d57.el9 ceph 1.5 M 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 142 k 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 3.5 M 2024-04-17T06:07:03.050 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 7.4 M 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 49 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:19.0.0-2862.g61531d57.el9 ceph 10 M 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:19.0.0-2862.g61531d57.el9 ceph 49 M 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 264 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 762 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:19.0.0-2862.g61531d57.el9 ceph 225 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell x86_64 2:19.0.0-2862.g61531d57.el9 ceph 27 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 18 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:19.0.0-2862.g61531d57.el9 ceph 32 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 746 k 2024-04-17T06:07:03.051 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:19.0.0-2862.g61531d57.el9 ceph 127 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:19.0.0-2862.g61531d57.el9 ceph 161 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:19.0.0-2862.g61531d57.el9 ceph 321 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:19.0.0-2862.g61531d57.el9 ceph 297 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:19.0.0-2862.g61531d57.el9 ceph 99 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:19.0.0-2862.g61531d57.el9 ceph 85 k 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:19.0.0-2862.g61531d57.el9 ceph 3.1 M 2024-04-17T06:07:03.052 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:19.0.0-2862.g61531d57.el9 ceph 170 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-17T06:07:03.053 INFO:teuthology.orchestra.run.smithi190.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 3.4 M 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 3.1 M 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-17T06:07:03.054 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:19.0.0-2862.g61531d57.el9 ceph 21 M 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 34 k 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:19.0.0-2862.g61531d57.el9 ceph 2.4 M 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 249 k 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:19.0.0-2862.g61531d57.el9 ceph 4.7 M 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:19.0.0-2862.g61531d57.el9 ceph 17 M 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2862.g61531d57.el9 ceph-noarch 16 k 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:19.0.0-2862.g61531d57.el9 ceph 25 k 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-17T06:07:03.055 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:19.0.0-2862.g61531d57.el9 ceph 163 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-17T06:07:03.056 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 501 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:19.0.0-2862.g61531d57.el9 ceph 5.2 M 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-17T06:07:03.057 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:19.0.0-2862.g61531d57.el9 ceph 45 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:19.0.0-2862.g61531d57.el9 ceph 135 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-17T06:07:03.058 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-17T06:07:03.059 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-17T06:07:03.060 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-17T06:07:03.061 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-17T06:07:03.062 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-17T06:07:03.063 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2024-04-17T06:07:03.063 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-17T06:07:03.063 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:03.063 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2024-04-17T06:07:03.063 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2024-04-17T06:07:03.063 INFO:teuthology.orchestra.run.smithi190.stdout:Install 150 Packages 2024-04-17T06:07:03.063 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 24 Packages 2024-04-17T06:07:03.063 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:03.064 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 228 M 2024-04-17T06:07:03.064 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2024-04-17T06:07:03.958 INFO:teuthology.orchestra.run.smithi190.stdout:(1/174): ceph-19.0.0-2862.g61531d57.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-17T06:07:04.257 INFO:teuthology.orchestra.run.smithi190.stdout:(2/174): ceph-fuse-19.0.0-2862.g61531d57.el9.x8 2.9 MB/s | 891 kB 00:00 2024-04-17T06:07:04.374 INFO:teuthology.orchestra.run.smithi190.stdout:(3/174): ceph-immutable-object-cache-19.0.0-286 1.2 MB/s | 145 kB 00:00 2024-04-17T06:07:04.458 INFO:teuthology.orchestra.run.smithi190.stdout:(4/174): ceph-base-19.0.0-2862.g61531d57.el9.x8 7.5 MB/s | 5.4 MB 00:00 2024-04-17T06:07:04.641 INFO:teuthology.orchestra.run.smithi190.stdout:(5/174): ceph-mds-19.0.0-2862.g61531d57.el9.x86 8.9 MB/s | 2.4 MB 00:00 2024-04-17T06:07:05.017 INFO:teuthology.orchestra.run.smithi190.stdout:(6/174): ceph-mon-19.0.0-2862.g61531d57.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-17T06:07:05.038 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-04-17T06:07:05.038 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-04-17T06:07:05.300 INFO:teuthology.orchestra.run.smithi190.stdout:(7/174): ceph-mgr-19.0.0-2862.g61531d57.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-04-17T06:07:06.375 INFO:teuthology.orchestra.run.smithi190.stdout:(8/174): ceph-common-19.0.0-2862.g61531d57.el9. 7.9 MB/s | 21 MB 00:02 2024-04-17T06:07:06.542 INFO:teuthology.orchestra.run.smithi190.stdout:(9/174): ceph-osd-19.0.0-2862.g61531d57.el9.x86 11 MB/s | 17 MB 00:01 2024-04-17T06:07:06.651 INFO:teuthology.orchestra.run.smithi190.stdout:(10/174): ceph-radosgw-19.0.0-2862.g61531d57.el 7.4 MB/s | 10 MB 00:01 2024-04-17T06:07:06.692 INFO:teuthology.orchestra.run.smithi190.stdout:(11/174): ceph-selinux-19.0.0-2862.g61531d57.el 79 kB/s | 25 kB 00:00 2024-04-17T06:07:06.726 INFO:teuthology.orchestra.run.smithi190.stdout:(12/174): cephfs-mirror-19.0.0-2862.g61531d57.e 2.9 MB/s | 225 kB 00:00 2024-04-17T06:07:06.793 INFO:teuthology.orchestra.run.smithi190.stdout:(13/174): libcephfs-devel-19.0.0-2862.g61531d57 476 kB/s | 32 kB 00:00 2024-04-17T06:07:06.994 INFO:teuthology.orchestra.run.smithi190.stdout:(14/174): cephfs-shell-19.0.0-2862.g61531d57.el 90 kB/s | 27 kB 00:00 2024-04-17T06:07:07.110 INFO:teuthology.orchestra.run.smithi190.stdout:(15/174): libcephsqlite-19.0.0-2862.g61531d57.e 1.4 MB/s | 163 kB 00:00 2024-04-17T06:07:07.203 INFO:teuthology.orchestra.run.smithi190.stdout:(16/174): librados-devel-19.0.0-2862.g61531d57. 1.4 MB/s | 127 kB 00:00 2024-04-17T06:07:07.244 INFO:teuthology.orchestra.run.smithi190.stdout:(17/174): libcephfs2-19.0.0-2862.g61531d57.el9. 1.6 MB/s | 746 kB 00:00 2024-04-17T06:07:07.361 INFO:teuthology.orchestra.run.smithi190.stdout:(18/174): libradosstriper1-19.0.0-2862.g61531d5 3.1 MB/s | 501 kB 00:00 2024-04-17T06:07:07.453 INFO:teuthology.orchestra.run.smithi190.stdout:(19/174): python3-ceph-argparse-19.0.0-2862.g61 493 kB/s | 45 kB 00:00 2024-04-17T06:07:07.553 INFO:teuthology.orchestra.run.smithi190.stdout:(20/174): python3-ceph-common-19.0.0-2862.g6153 1.3 MB/s | 135 kB 00:00 2024-04-17T06:07:07.662 INFO:teuthology.orchestra.run.smithi190.stdout:(21/174): python3-cephfs-19.0.0-2862.g61531d57. 1.5 MB/s | 161 kB 00:00 2024-04-17T06:07:07.795 INFO:teuthology.orchestra.run.smithi190.stdout:(22/174): librgw2-19.0.0-2862.g61531d57.el9.x86 9.5 MB/s | 5.2 MB 00:00 2024-04-17T06:07:07.820 INFO:teuthology.orchestra.run.smithi190.stdout:(23/174): python3-rados-19.0.0-2862.g61531d57.e 2.0 MB/s | 321 kB 00:00 2024-04-17T06:07:07.871 INFO:teuthology.orchestra.run.smithi190.stdout:(24/174): python3-rbd-19.0.0-2862.g61531d57.el9 3.9 MB/s | 297 kB 00:00 2024-04-17T06:07:07.905 INFO:teuthology.orchestra.run.smithi190.stdout:(25/174): python3-rgw-19.0.0-2862.g61531d57.el9 1.2 MB/s | 99 kB 00:00 2024-04-17T06:07:07.947 INFO:teuthology.orchestra.run.smithi190.stdout:(26/174): rbd-fuse-19.0.0-2862.g61531d57.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-17T06:07:08.039 INFO:teuthology.orchestra.run.smithi190.stdout:(27/174): rbd-nbd-19.0.0-2862.g61531d57.el9.x86 1.8 MB/s | 170 kB 00:00 2024-04-17T06:07:08.122 INFO:teuthology.orchestra.run.smithi190.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2862.g 411 kB/s | 34 kB 00:00 2024-04-17T06:07:08.214 INFO:teuthology.orchestra.run.smithi190.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2862.g61531d5 1.5 MB/s | 142 kB 00:00 2024-04-17T06:07:08.331 INFO:teuthology.orchestra.run.smithi190.stdout:(30/174): rbd-mirror-19.0.0-2862.g61531d57.el9. 7.3 MB/s | 3.1 MB 00:00 2024-04-17T06:07:08.698 INFO:teuthology.orchestra.run.smithi190.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2862.g61531 7.3 MB/s | 3.5 MB 00:00 2024-04-17T06:07:08.799 INFO:teuthology.orchestra.run.smithi190.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2862.g61 2.4 MB/s | 249 kB 00:00 2024-04-17T06:07:08.890 INFO:teuthology.orchestra.run.smithi190.stdout:(33/174): ceph-mgr-rook-19.0.0-2862.g61531d57.e 538 kB/s | 49 kB 00:00 2024-04-17T06:07:08.982 INFO:teuthology.orchestra.run.smithi190.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2862.g6 180 kB/s | 16 kB 00:00 2024-04-17T06:07:09.083 INFO:teuthology.orchestra.run.smithi190.stdout:(35/174): ceph-volume-19.0.0-2862.g61531d57.el9 2.6 MB/s | 264 kB 00:00 2024-04-17T06:07:09.558 INFO:teuthology.orchestra.run.smithi190.stdout:(36/174): ceph-test-19.0.0-2862.g61531d57.el9.x 16 MB/s | 49 MB 00:03 2024-04-17T06:07:09.600 INFO:teuthology.orchestra.run.smithi190.stdout:(37/174): cephadm-19.0.0-2862.g61531d57.el9.noa 1.4 MB/s | 762 kB 00:00 2024-04-17T06:07:09.625 INFO:teuthology.orchestra.run.smithi190.stdout:(38/174): cephfs-top-19.0.0-2862.g61531d57.el9. 274 kB/s | 18 kB 00:00 2024-04-17T06:07:09.717 INFO:teuthology.orchestra.run.smithi190.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-04-17T06:07:09.875 INFO:teuthology.orchestra.run.smithi190.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 987 kB/s | 106 kB 00:00 2024-04-17T06:07:09.909 INFO:teuthology.orchestra.run.smithi190.stdout:(41/174): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2024-04-17T06:07:09.934 INFO:teuthology.orchestra.run.smithi190.stdout:(42/174): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 662 kB 00:00 2024-04-17T06:07:09.959 INFO:teuthology.orchestra.run.smithi190.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-04-17T06:07:09.976 INFO:teuthology.orchestra.run.smithi190.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 474 kB/s | 32 kB 00:00 2024-04-17T06:07:10.085 INFO:teuthology.orchestra.run.smithi190.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 137 kB/s | 17 kB 00:00 2024-04-17T06:07:10.102 INFO:teuthology.orchestra.run.smithi190.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 81 kB/s | 10 kB 00:00 2024-04-17T06:07:10.152 INFO:teuthology.orchestra.run.smithi190.stdout:(47/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-17T06:07:10.177 INFO:teuthology.orchestra.run.smithi190.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-04-17T06:07:10.194 INFO:teuthology.orchestra.run.smithi190.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 484 kB/s | 44 kB 00:00 2024-04-17T06:07:10.219 INFO:teuthology.orchestra.run.smithi190.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-04-17T06:07:10.236 INFO:teuthology.orchestra.run.smithi190.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-17T06:07:10.261 INFO:teuthology.orchestra.run.smithi190.stdout:(52/174): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 578 kB 00:00 2024-04-17T06:07:10.286 INFO:teuthology.orchestra.run.smithi190.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2024-04-17T06:07:10.312 INFO:teuthology.orchestra.run.smithi190.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-04-17T06:07:10.337 INFO:teuthology.orchestra.run.smithi190.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-04-17T06:07:10.354 INFO:teuthology.orchestra.run.smithi190.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 822 kB/s | 34 kB 00:00 2024-04-17T06:07:10.496 INFO:teuthology.orchestra.run.smithi190.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-17T06:07:10.538 INFO:teuthology.orchestra.run.smithi190.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 7.7 MB/s | 1.5 MB 00:00 2024-04-17T06:07:10.572 INFO:teuthology.orchestra.run.smithi190.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 817 kB 00:00 2024-04-17T06:07:10.597 INFO:teuthology.orchestra.run.smithi190.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 132 kB 00:00 2024-04-17T06:07:10.630 INFO:teuthology.orchestra.run.smithi190.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-04-17T06:07:10.656 INFO:teuthology.orchestra.run.smithi190.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-17T06:07:10.789 INFO:teuthology.orchestra.run.smithi190.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 37 MB/s | 5.8 MB 00:00 2024-04-17T06:07:10.806 INFO:teuthology.orchestra.run.smithi190.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-04-17T06:07:10.890 INFO:teuthology.orchestra.run.smithi190.stdout:(65/174): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.7 MB 00:00 2024-04-17T06:07:10.906 INFO:teuthology.orchestra.run.smithi190.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 157 kB/s | 18 kB 00:00 2024-04-17T06:07:10.932 INFO:teuthology.orchestra.run.smithi190.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2024-04-17T06:07:10.949 INFO:teuthology.orchestra.run.smithi190.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 543 kB/s | 32 kB 00:00 2024-04-17T06:07:10.990 INFO:teuthology.orchestra.run.smithi190.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.6 MB/s | 217 kB 00:00 2024-04-17T06:07:11.074 INFO:teuthology.orchestra.run.smithi190.stdout:(70/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-04-17T06:07:11.100 INFO:teuthology.orchestra.run.smithi190.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-04-17T06:07:11.125 INFO:teuthology.orchestra.run.smithi190.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 522 kB/s | 69 kB 00:00 2024-04-17T06:07:11.150 INFO:teuthology.orchestra.run.smithi190.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 550 kB/s | 41 kB 00:00 2024-04-17T06:07:11.175 INFO:teuthology.orchestra.run.smithi190.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-17T06:07:11.200 INFO:teuthology.orchestra.run.smithi190.stdout:(75/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-17T06:07:11.226 INFO:teuthology.orchestra.run.smithi190.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-17T06:07:11.251 INFO:teuthology.orchestra.run.smithi190.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 634 kB/s | 47 kB 00:00 2024-04-17T06:07:11.276 INFO:teuthology.orchestra.run.smithi190.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-04-17T06:07:11.560 INFO:teuthology.orchestra.run.smithi190.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-04-17T06:07:11.585 INFO:teuthology.orchestra.run.smithi190.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 116 kB/s | 39 kB 00:00 2024-04-17T06:07:11.610 INFO:teuthology.orchestra.run.smithi190.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 123 kB/s | 41 kB 00:00 2024-04-17T06:07:11.635 INFO:teuthology.orchestra.run.smithi190.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-04-17T06:07:11.661 INFO:teuthology.orchestra.run.smithi190.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 931 kB/s | 46 kB 00:00 2024-04-17T06:07:11.686 INFO:teuthology.orchestra.run.smithi190.stdout:(84/174): qatlib-service-24.02.0-1.el9.x86_64.r 464 kB/s | 35 kB 00:00 2024-04-17T06:07:11.711 INFO:teuthology.orchestra.run.smithi190.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-17T06:07:11.736 INFO:teuthology.orchestra.run.smithi190.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 752 kB/s | 56 kB 00:00 2024-04-17T06:07:11.761 INFO:teuthology.orchestra.run.smithi190.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-17T06:07:11.786 INFO:teuthology.orchestra.run.smithi190.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 586 kB/s | 44 kB 00:00 2024-04-17T06:07:11.820 INFO:teuthology.orchestra.run.smithi190.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 944 kB/s | 79 kB 00:00 2024-04-17T06:07:11.845 INFO:teuthology.orchestra.run.smithi190.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-04-17T06:07:11.879 INFO:teuthology.orchestra.run.smithi190.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-04-17T06:07:11.904 INFO:teuthology.orchestra.run.smithi190.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-04-17T06:07:11.930 INFO:teuthology.orchestra.run.smithi190.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-04-17T06:07:12.022 INFO:teuthology.orchestra.run.smithi190.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 187 kB 00:00 2024-04-17T06:07:12.103 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-04-17T06:07:12.189 INFO:teuthology.orchestra.run.smithi190.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 659 kB/s | 187 kB 00:00 2024-04-17T06:07:12.189 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T06:07:12.214 INFO:teuthology.orchestra.run.smithi190.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-17T06:07:12.232 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T06:07:12.247 INFO:teuthology.orchestra.run.smithi190.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-17T06:07:12.271 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:07:12.281 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:07:12.314 INFO:teuthology.orchestra.run.smithi190.stdout:(98/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-04-17T06:07:12.322 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T06:07:12.340 INFO:teuthology.orchestra.run.smithi190.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 171 kB/s | 21 kB 00:00 2024-04-17T06:07:12.365 INFO:teuthology.orchestra.run.smithi190.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 217 kB/s | 25 kB 00:00 2024-04-17T06:07:12.375 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T06:07:12.407 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T06:07:12.434 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T06:07:12.478 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T06:07:12.499 INFO:teuthology.orchestra.run.smithi190.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.8 MB/s | 248 kB 00:00 2024-04-17T06:07:12.524 INFO:teuthology.orchestra.run.smithi190.stdout:(102/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.3 MB/s | 245 kB 00:00 2024-04-17T06:07:12.549 INFO:teuthology.orchestra.run.smithi190.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-17T06:07:12.674 INFO:teuthology.orchestra.run.smithi190.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 11 MB/s | 4.1 MB 00:00 2024-04-17T06:07:12.791 INFO:teuthology.orchestra.run.smithi190.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-04-17T06:07:12.813 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T06:07:12.876 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:19.0.0-2862.g61531d57.el9.x86_64 10/198 2024-04-17T06:07:13.006 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:19.0.0-2862.g61531d57.el9.x86_64 10/198 2024-04-17T06:07:13.043 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:19.0.0-2862.g61531d57.el9.x86_64 11/198 2024-04-17T06:07:13.116 INFO:teuthology.orchestra.run.smithi190.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-17T06:07:13.133 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:19.0.0-2862.g61531d57.el9.x86_64 11/198 2024-04-17T06:07:13.173 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:19.0.0-2862.g61531d57.el9.x86_64 12/198 2024-04-17T06:07:13.183 INFO:teuthology.orchestra.run.smithi190.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 788 kB/s | 309 kB 00:00 2024-04-17T06:07:13.200 INFO:teuthology.orchestra.run.smithi190.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 234 kB/s | 19 kB 00:00 2024-04-17T06:07:13.209 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T06:07:13.267 INFO:teuthology.orchestra.run.smithi190.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-04-17T06:07:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 731 kB/s | 49 kB 00:00 2024-04-17T06:07:13.359 INFO:teuthology.orchestra.run.smithi190.stdout:(111/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-04-17T06:07:13.399 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T06:07:13.401 INFO:teuthology.orchestra.run.smithi190.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-17T06:07:13.453 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T06:07:13.481 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:19.0.0-2862.g61531d57.el9.x86_64 16/198 2024-04-17T06:07:13.493 INFO:teuthology.orchestra.run.smithi190.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-17T06:07:13.527 INFO:teuthology.orchestra.run.smithi190.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-04-17T06:07:13.567 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2862.g61531d57.el9.x86_64 16/198 2024-04-17T06:07:13.577 INFO:teuthology.orchestra.run.smithi190.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-04-17T06:07:13.590 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:19.0.0-2862.g61531d57.el9.x86 17/198 2024-04-17T06:07:13.602 INFO:teuthology.orchestra.run.smithi190.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 577 kB/s | 43 kB 00:00 2024-04-17T06:07:13.644 INFO:teuthology.orchestra.run.smithi190.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-04-17T06:07:13.656 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2862.g61531d57.el9.x86 17/198 2024-04-17T06:07:13.669 INFO:teuthology.orchestra.run.smithi190.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-17T06:07:13.686 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T06:07:13.709 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:07:13.711 INFO:teuthology.orchestra.run.smithi190.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-17T06:07:13.725 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:07:13.736 INFO:teuthology.orchestra.run.smithi190.stdout:(120/174): luarocks-3.9.2-1.el9.noarch.rpm 401 kB/s | 151 kB 00:00 2024-04-17T06:07:13.761 INFO:teuthology.orchestra.run.smithi190.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-17T06:07:13.787 INFO:teuthology.orchestra.run.smithi190.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-17T06:07:13.820 INFO:teuthology.orchestra.run.smithi190.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-04-17T06:07:13.846 INFO:teuthology.orchestra.run.smithi190.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-17T06:07:13.896 INFO:teuthology.orchestra.run.smithi190.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-17T06:07:13.928 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:07:13.930 INFO:teuthology.orchestra.run.smithi190.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-04-17T06:07:13.955 INFO:teuthology.orchestra.run.smithi190.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-17T06:07:13.967 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:07:13.980 INFO:teuthology.orchestra.run.smithi190.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 210 kB/s | 18 kB 00:00 2024-04-17T06:07:14.005 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T06:07:14.025 INFO:teuthology.orchestra.run.smithi190.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 244 kB/s | 23 kB 00:00 2024-04-17T06:07:14.056 INFO:teuthology.orchestra.run.smithi190.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-04-17T06:07:14.081 INFO:teuthology.orchestra.run.smithi190.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-17T06:07:14.115 INFO:teuthology.orchestra.run.smithi190.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-17T06:07:14.149 INFO:teuthology.orchestra.run.smithi190.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 695 kB/s | 46 kB 00:00 2024-04-17T06:07:14.173 INFO:teuthology.orchestra.run.smithi190.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 860 kB/s | 79 kB 00:00 2024-04-17T06:07:14.188 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T06:07:14.199 INFO:teuthology.orchestra.run.smithi190.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-17T06:07:14.224 INFO:teuthology.orchestra.run.smithi190.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-17T06:07:14.258 INFO:teuthology.orchestra.run.smithi190.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-17T06:07:14.283 INFO:teuthology.orchestra.run.smithi190.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-17T06:07:14.308 INFO:teuthology.orchestra.run.smithi190.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 336 kB/s | 28 kB 00:00 2024-04-17T06:07:14.326 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:07:14.333 INFO:teuthology.orchestra.run.smithi190.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-17T06:07:14.367 INFO:teuthology.orchestra.run.smithi190.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-17T06:07:14.386 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:07:14.392 INFO:teuthology.orchestra.run.smithi190.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-17T06:07:14.417 INFO:teuthology.orchestra.run.smithi190.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-04-17T06:07:14.443 INFO:teuthology.orchestra.run.smithi190.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-04-17T06:07:14.468 INFO:teuthology.orchestra.run.smithi190.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-17T06:07:14.480 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:07:14.501 INFO:teuthology.orchestra.run.smithi190.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-17T06:07:14.527 INFO:teuthology.orchestra.run.smithi190.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-17T06:07:14.529 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:07:14.552 INFO:teuthology.orchestra.run.smithi190.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-17T06:07:14.611 INFO:teuthology.orchestra.run.smithi190.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-17T06:07:14.678 INFO:teuthology.orchestra.run.smithi190.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-17T06:07:14.742 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:07:14.851 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T06:07:14.886 INFO:teuthology.orchestra.run.smithi190.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-04-17T06:07:14.946 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T06:07:14.953 INFO:teuthology.orchestra.run.smithi190.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-04-17T06:07:14.976 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T06:07:14.978 INFO:teuthology.orchestra.run.smithi190.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-04-17T06:07:15.002 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T06:07:15.020 INFO:teuthology.orchestra.run.smithi190.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-04-17T06:07:15.046 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T06:07:15.062 INFO:teuthology.orchestra.run.smithi190.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-04-17T06:07:15.121 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T06:07:15.146 INFO:teuthology.orchestra.run.smithi190.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 857 kB/s | 71 kB 00:00 2024-04-17T06:07:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T06:07:15.204 INFO:teuthology.orchestra.run.smithi190.stdout:(157/174): librados2-19.0.0-2862.g61531d57.el9. 5.2 MB/s | 3.4 MB 00:00 2024-04-17T06:07:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T06:07:15.238 INFO:teuthology.orchestra.run.smithi190.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 396 kB/s | 36 kB 00:00 2024-04-17T06:07:15.285 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T06:07:15.288 INFO:teuthology.orchestra.run.smithi190.stdout:(159/174): librbd1-19.0.0-2862.g61531d57.el9.x8 4.8 MB/s | 3.1 MB 00:00 2024-04-17T06:07:15.320 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T06:07:15.322 INFO:teuthology.orchestra.run.smithi190.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-04-17T06:07:15.363 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T06:07:15.364 INFO:teuthology.orchestra.run.smithi190.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 306 kB/s | 23 kB 00:00 2024-04-17T06:07:15.381 INFO:teuthology.orchestra.run.smithi190.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-04-17T06:07:15.397 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T06:07:15.406 INFO:teuthology.orchestra.run.smithi190.stdout:(163/174): elfutils-libelf-0.190-2.el9.x86_64.r 953 kB/s | 192 kB 00:00 2024-04-17T06:07:15.423 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T06:07:15.431 INFO:teuthology.orchestra.run.smithi190.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-17T06:07:15.456 INFO:teuthology.orchestra.run.smithi190.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-17T06:07:15.482 INFO:teuthology.orchestra.run.smithi190.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 570 kB/s | 29 kB 00:00 2024-04-17T06:07:15.502 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T06:07:15.507 INFO:teuthology.orchestra.run.smithi190.stdout:(167/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 155 kB 00:00 2024-04-17T06:07:15.537 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:19.0.0-2862.g61531d57.el9. 39/198 2024-04-17T06:07:15.580 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:19.0.0-2862.g61531d57.el 40/198 2024-04-17T06:07:15.600 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:19.0.0-2862.g61531d57.el9.x86_6 41/198 2024-04-17T06:07:15.679 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T06:07:15.699 INFO:teuthology.orchestra.run.smithi190.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 5.0 MB/s | 1.2 MB 00:00 2024-04-17T06:07:15.724 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T06:07:15.800 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T06:07:15.931 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T06:07:15.932 INFO:teuthology.orchestra.run.smithi190.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-04-17T06:07:15.932 INFO:teuthology.orchestra.run.smithi190.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 5.9 MB/s | 2.2 MB 00:00 2024-04-17T06:07:15.932 INFO:teuthology.orchestra.run.smithi190.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 533 kB/s | 31 kB 00:00 2024-04-17T06:07:15.932 INFO:teuthology.orchestra.run.smithi190.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.5 MB/s | 2.3 MB 00:00 2024-04-17T06:07:15.983 INFO:teuthology.orchestra.run.smithi190.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 684 kB/s | 74 kB 00:00 2024-04-17T06:07:15.984 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T06:07:16.075 INFO:teuthology.orchestra.run.smithi190.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.8 MB/s | 1.1 MB 00:00 2024-04-17T06:07:16.083 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2024-04-17T06:07:16.083 INFO:teuthology.orchestra.run.smithi190.stdout:Total 17 MB/s | 228 MB 00:13 2024-04-17T06:07:16.476 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2024-04-17T06:07:16.502 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T06:07:16.581 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T06:07:16.665 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T06:07:16.708 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:07:16.718 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:07:16.750 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2024-04-17T06:07:16.750 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2024-04-17T06:07:16.755 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T06:07:16.797 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T06:07:16.835 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:19.0.0-2862.g61531d57.el9.x86_6 53/198 2024-04-17T06:07:16.865 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T06:07:17.129 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T06:07:17.319 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T06:07:17.391 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T06:07:17.436 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T06:07:17.588 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T06:07:17.697 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T06:07:17.963 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T06:07:18.044 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T06:07:18.081 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T06:07:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T06:07:18.165 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T06:07:18.200 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T06:07:18.244 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T06:07:18.286 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T06:07:18.421 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T06:07:18.474 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T06:07:18.504 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T06:07:18.545 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T06:07:18.646 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T06:07:18.696 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T06:07:18.736 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T06:07:18.752 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2024-04-17T06:07:18.752 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2024-04-17T06:07:18.762 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T06:07:18.786 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T06:07:18.874 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T06:07:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T06:07:19.093 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T06:07:19.136 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T06:07:19.226 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T06:07:19.282 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T06:07:19.467 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T06:07:19.515 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T06:07:19.645 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T06:07:19.767 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T06:07:19.940 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T06:07:20.062 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:07:20.230 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T06:07:20.264 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T06:07:20.303 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T06:07:20.335 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T06:07:20.364 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T06:07:20.459 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T06:07:20.491 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T06:07:20.525 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T06:07:20.557 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T06:07:20.592 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T06:07:20.683 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T06:07:20.771 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T06:07:20.815 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T06:07:20.846 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T06:07:20.894 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T06:07:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T06:07:21.047 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T06:07:21.777 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T06:07:21.828 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T06:07:21.862 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T06:07:21.911 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T06:07:21.998 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T06:07:22.030 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T06:07:22.207 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T06:07:22.244 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T06:07:22.323 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T06:07:22.969 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T06:07:23.092 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T06:07:24.331 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T06:07:24.428 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T06:07:24.459 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T06:07:24.482 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T06:07:24.510 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T06:07:24.826 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T06:07:24.856 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:19.0.0-2862.g61531d57.el9.x86_64 124/198 2024-04-17T06:07:24.946 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:19.0.0-2862.g61531d57.el9.x86_64 124/198 2024-04-17T06:07:24.977 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:19.0.0-2862.g61531d57.el9.x86_64 125/198 2024-04-17T06:07:25.014 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T06:07:25.489 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T06:07:25.518 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:19.0.0-2862.g61531d57.el9.x86_64 128/198 2024-04-17T06:07:25.588 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:19.0.0-2862.g61531d57.el9.x86_64 128/198 2024-04-17T06:07:25.611 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:19.0.0-2862.g61531d57.el9.x86_64 129/198 2024-04-17T06:07:25.642 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2024-04-17T06:07:25.730 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-17T06:07:25.773 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-17T06:07:25.819 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:07:25.830 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-17T06:07:25.869 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-17T06:07:25.936 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-17T06:07:25.972 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-17T06:07:26.007 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-17T06:07:26.102 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-17T06:07:26.435 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-17T06:07:26.500 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:19.0.0-2862.g61531d57.el9.x86_64 10/198 2024-04-17T06:07:26.620 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:19.0.0-2862.g61531d57.el9.x86_64 10/198 2024-04-17T06:07:26.642 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:19.0.0-2862.g61531d57.el9.x86_64 11/198 2024-04-17T06:07:26.724 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:19.0.0-2862.g61531d57.el9.x86_64 11/198 2024-04-17T06:07:26.764 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:19.0.0-2862.g61531d57.el9.x86_64 12/198 2024-04-17T06:07:26.802 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-17T06:07:26.987 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-17T06:07:27.035 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-17T06:07:27.063 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:19.0.0-2862.g61531d57.el9.x86_64 16/198 2024-04-17T06:07:27.166 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2862.g61531d57.el9.x86_64 16/198 2024-04-17T06:07:27.189 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:19.0.0-2862.g61531d57.el9.x86 17/198 2024-04-17T06:07:27.238 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2862.g61531d57.el9.x86 17/198 2024-04-17T06:07:27.277 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-17T06:07:27.299 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:07:27.316 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-17T06:07:27.510 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:07:27.557 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-17T06:07:27.587 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-17T06:07:27.750 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-17T06:07:27.873 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 130/198 2024-04-17T06:07:27.907 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 130/198 2024-04-17T06:07:27.972 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:07:28.015 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-17T06:07:28.110 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:07:28.166 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:07:28.387 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-17T06:07:28.458 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 130/198 2024-04-17T06:07:28.486 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-17T06:07:28.493 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:19.0.0-2862.g61531d57.el9.x86_64 131/198 2024-04-17T06:07:28.573 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-17T06:07:28.585 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:19.0.0-2862.g61531d57.el9.x86_64 131/198 2024-04-17T06:07:28.585 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T06:07:28.585 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T06:07:28.585 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:28.605 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-17T06:07:28.620 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:19.0.0-2862.g61531d57.el9.x86_64 132/198 2024-04-17T06:07:28.631 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-17T06:07:28.675 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-17T06:07:28.749 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-17T06:07:28.803 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-17T06:07:28.841 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-17T06:07:28.905 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-17T06:07:28.940 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-17T06:07:28.983 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-17T06:07:29.017 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-17T06:07:29.044 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-17T06:07:29.113 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-17T06:07:29.149 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:19.0.0-2862.g61531d57.el9. 39/198 2024-04-17T06:07:29.192 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:19.0.0-2862.g61531d57.el 40/198 2024-04-17T06:07:29.221 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:19.0.0-2862.g61531d57.el9.x86_6 41/198 2024-04-17T06:07:29.290 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-17T06:07:29.343 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-17T06:07:29.417 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-17T06:07:29.550 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-17T06:07:29.604 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-17T06:07:30.119 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-17T06:07:30.199 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-17T06:07:30.281 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-17T06:07:30.330 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:07:30.340 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-17T06:07:30.382 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-17T06:07:30.427 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-17T06:07:30.465 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:19.0.0-2862.g61531d57.el9.x86_6 53/198 2024-04-17T06:07:30.496 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-17T06:07:30.783 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-17T06:07:30.980 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-17T06:07:31.064 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-17T06:07:31.107 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-17T06:07:31.247 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-17T06:07:31.294 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-17T06:07:31.348 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-17T06:07:31.413 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-17T06:07:31.605 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-17T06:07:31.642 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-17T06:07:31.688 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-17T06:07:31.748 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-17T06:07:31.792 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-17T06:07:31.842 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-17T06:07:31.975 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-17T06:07:32.031 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-17T06:07:32.061 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-17T06:07:32.102 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-17T06:07:32.197 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-17T06:07:32.228 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-17T06:07:32.260 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-17T06:07:32.277 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-17T06:07:32.302 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-17T06:07:32.381 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-17T06:07:32.511 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-17T06:07:32.626 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-17T06:07:32.702 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-17T06:07:32.750 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-17T06:07:32.806 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-17T06:07:32.989 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-17T06:07:33.040 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-17T06:07:33.175 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-17T06:07:33.213 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-17T06:07:33.245 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-17T06:07:33.276 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:07:33.295 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-17T06:07:33.322 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-17T06:07:33.361 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-17T06:07:33.392 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-17T06:07:33.422 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-17T06:07:33.499 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-17T06:07:33.532 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-17T06:07:33.566 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-17T06:07:33.607 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-17T06:07:33.642 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-17T06:07:33.730 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-17T06:07:33.812 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-17T06:07:33.856 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-17T06:07:33.888 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-17T06:07:33.935 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-17T06:07:34.031 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-17T06:07:34.080 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-17T06:07:34.811 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-17T06:07:34.848 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-17T06:07:34.882 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-17T06:07:34.930 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-17T06:07:35.012 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-17T06:07:35.049 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-17T06:07:35.225 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-17T06:07:35.272 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-17T06:07:35.350 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-17T06:07:35.984 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-17T06:07:36.093 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-17T06:07:37.308 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-17T06:07:37.390 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-17T06:07:37.421 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-17T06:07:37.453 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-17T06:07:37.480 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-17T06:07:37.790 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-17T06:07:37.818 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:19.0.0-2862.g61531d57.el9.x86_64 124/198 2024-04-17T06:07:37.909 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:19.0.0-2862.g61531d57.el9.x86_64 124/198 2024-04-17T06:07:37.940 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:19.0.0-2862.g61531d57.el9.x86_64 125/198 2024-04-17T06:07:37.976 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-17T06:07:38.441 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-17T06:07:38.471 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:19.0.0-2862.g61531d57.el9.x86_64 128/198 2024-04-17T06:07:38.526 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:19.0.0-2862.g61531d57.el9.x86_64 128/198 2024-04-17T06:07:38.548 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:19.0.0-2862.g61531d57.el9.x86_64 129/198 2024-04-17T06:07:40.808 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 130/198 2024-04-17T06:07:40.836 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 130/198 2024-04-17T06:07:40.994 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2862.g61531d57.el9.x86_64 132/198 2024-04-17T06:07:40.994 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-04-17T06:07:40.994 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-04-17T06:07:40.994 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-04-17T06:07:40.994 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-04-17T06:07:40.994 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-04-17T06:07:40.994 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-04-17T06:07:40.994 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-04-17T06:07:40.995 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:41.166 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:19.0.0-2862.g61531d57.el9.noarch 133/198 2024-04-17T06:07:41.182 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:19.0.0-2862.g61531d57.el9.noarch 133/198 2024-04-17T06:07:41.183 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:41.183 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T06:07:41.183 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:41.378 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 130/198 2024-04-17T06:07:41.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:19.0.0-2862.g61531d57.el9.x86_64 131/198 2024-04-17T06:07:41.551 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:19.0.0-2862.g61531d57.el9.x86_64 131/198 2024-04-17T06:07:41.551 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-17T06:07:41.551 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-17T06:07:41.551 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:41.551 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:19.0.0-2862.g61531d57.el9.x86_64 132/198 2024-04-17T06:07:42.678 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:19.0.0-2862.g61531d57.el9.x86_64 134/198 2024-04-17T06:07:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:19.0.0-2862.g61531d57.el9.x86_64 134/198 2024-04-17T06:07:42.705 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:42.705 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T06:07:42.705 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:07:42.705 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:07:42.705 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:42.928 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:19.0.0-2862.g61531d57.el9.x86_64 135/198 2024-04-17T06:07:42.951 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:19.0.0-2862.g61531d57.el9.x86_64 135/198 2024-04-17T06:07:42.952 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:42.952 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T06:07:42.952 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:07:42.952 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:07:42.952 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:43.387 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:19.0.0-2862.g61531d57.el9.x86_64 136/198 2024-04-17T06:07:43.410 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:19.0.0-2862.g61531d57.el9.x86_64 136/198 2024-04-17T06:07:43.410 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:43.410 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T06:07:43.410 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:07:43.410 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:07:43.411 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:43.505 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T06:07:43.959 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:19.0.0-2862.g61531d57.el9.noarch 138/198 2024-04-17T06:07:43.987 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:19.0.0-2862.g61531d57.el9.noarch 138/198 2024-04-17T06:07:44.019 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2862.g61531d57.e 139/198 2024-04-17T06:07:44.079 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2862.g61531d57. 140/198 2024-04-17T06:07:44.110 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2862.g61531d57.el9.noa 141/198 2024-04-17T06:07:45.151 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2862.g61531d57.el9.noa 141/198 2024-04-17T06:07:45.187 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2862.g61531d57.el9.n 142/198 2024-04-17T06:07:46.158 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2862.g61531d57.el9.n 142/198 2024-04-17T06:07:46.179 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2862.g615 143/198 2024-04-17T06:07:46.282 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2862.g615 143/198 2024-04-17T06:07:46.459 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2862.g61531d57.el 144/198 2024-04-17T06:07:46.489 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:19.0.0-2862.g61531d57.el9.x86_64 145/198 2024-04-17T06:07:46.514 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2862.g61531d57.el9.x86_64 145/198 2024-04-17T06:07:46.514 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:46.514 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T06:07:46.514 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:07:46.514 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:07:46.514 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:46.565 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:19.0.0-2862.g61531d57.el9.noarch 146/198 2024-04-17T06:07:46.577 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2862.g61531d57.el9.noarch 146/198 2024-04-17T06:07:46.622 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:19.0.0-2862.g61531d57.el9.x86_64 147/198 2024-04-17T06:07:47.494 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T06:07:47.525 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:19.0.0-2862.g61531d57.el9.x86_64 149/198 2024-04-17T06:07:47.549 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2862.g61531d57.el9.x86_64 149/198 2024-04-17T06:07:47.549 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:47.549 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T06:07:47.549 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:07:47.549 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:07:47.549 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:47.592 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2862.g61531 150/198 2024-04-17T06:07:47.616 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2862.g61531 150/198 2024-04-17T06:07:47.616 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:47.616 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T06:07:47.616 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:47.660 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 151/198 2024-04-17T06:07:47.683 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 151/198 2024-04-17T06:07:47.683 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:47.684 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T06:07:47.684 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:07:47.684 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:07:47.684 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:47.986 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 152/198 2024-04-17T06:07:48.009 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 152/198 2024-04-17T06:07:48.009 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:48.009 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T06:07:48.009 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:07:48.009 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:07:48.009 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:07:52.557 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:19.0.0-2862.g61531d57.el9.x86_64 153/198 2024-04-17T06:07:52.606 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 154/198 2024-04-17T06:07:52.653 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:19.0.0-2862.g61531d57.el9.x86_64 155/198 2024-04-17T06:07:52.680 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T06:07:52.863 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-shell-2:19.0.0-2862.g61531d57.el9.x86_64 157/198 2024-04-17T06:07:52.982 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T06:07:53.027 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 159/198 2024-04-17T06:07:53.089 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T06:07:53.145 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T06:07:53.188 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T06:07:53.221 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:19.0.0-2862.g61531d57.el9.x86_ 163/198 2024-04-17T06:07:53.231 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2862.g61531d57.el9.x86_64 132/198 2024-04-17T06:07:53.232 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2024-04-17T06:07:53.232 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2024-04-17T06:07:53.232 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2024-04-17T06:07:53.232 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2024-04-17T06:07:53.232 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2024-04-17T06:07:53.232 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2024-04-17T06:07:53.232 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2024-04-17T06:07:53.232 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:53.253 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T06:07:53.282 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T06:07:53.342 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T06:07:53.406 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:19.0.0-2862.g61531d57.el9.noarch 133/198 2024-04-17T06:07:53.413 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T06:07:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:19.0.0-2862.g61531d57.el9.noarch 133/198 2024-04-17T06:07:53.422 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:53.422 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-17T06:07:53.422 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:53.456 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T06:07:54.030 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:07:54.070 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:07:54.091 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:07:54.910 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:19.0.0-2862.g61531d57.el9.noarch 170/198 2024-04-17T06:07:54.917 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:19.0.0-2862.g61531d57.el9.x86_64 134/198 2024-04-17T06:07:54.945 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:19.0.0-2862.g61531d57.el9.x86_64 134/198 2024-04-17T06:07:54.945 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:54.945 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-17T06:07:54.945 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:07:54.945 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-17T06:07:54.945 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:55.029 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T06:07:55.075 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T06:07:55.165 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T06:07:55.168 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:19.0.0-2862.g61531d57.el9.x86_64 135/198 2024-04-17T06:07:55.191 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:19.0.0-2862.g61531d57.el9.x86_64 135/198 2024-04-17T06:07:55.191 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:55.191 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-17T06:07:55.191 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:07:55.191 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-17T06:07:55.192 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:55.196 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T06:07:55.221 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T06:07:55.246 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T06:07:55.273 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T06:07:55.273 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:07:55.361 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:07:55.390 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T06:07:55.390 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:07:55.458 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:07:55.484 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T06:07:55.526 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T06:07:55.551 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T06:07:55.576 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T06:07:55.611 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T06:07:55.635 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:19.0.0-2862.g61531d57.el9.x86_64 136/198 2024-04-17T06:07:55.659 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:19.0.0-2862.g61531d57.el9.x86_64 136/198 2024-04-17T06:07:55.659 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:55.660 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-17T06:07:55.660 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:07:55.660 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-17T06:07:55.660 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:55.666 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T06:07:55.698 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T06:07:55.723 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T06:07:55.748 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T06:07:55.754 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-17T06:07:55.774 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:07:55.807 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T06:07:55.814 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:07:55.814 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:07:55.993 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:07:55.993 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:07:56.109 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:07:56.141 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:07:56.168 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T06:07:56.182 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:19.0.0-2862.g61531d57.el9.noarch 138/198 2024-04-17T06:07:56.194 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T06:07:56.211 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:19.0.0-2862.g61531d57.el9.noarch 138/198 2024-04-17T06:07:56.227 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T06:07:56.243 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2862.g61531d57.e 139/198 2024-04-17T06:07:56.294 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:07:56.305 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2862.g61531d57. 140/198 2024-04-17T06:07:56.334 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2862.g61531d57.el9.noa 141/198 2024-04-17T06:07:57.368 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2862.g61531d57.el9.noa 141/198 2024-04-17T06:07:57.402 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2862.g61531d57.el9.n 142/198 2024-04-17T06:07:58.364 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2862.g61531d57.el9.n 142/198 2024-04-17T06:07:58.394 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2862.g615 143/198 2024-04-17T06:07:58.498 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2862.g615 143/198 2024-04-17T06:07:58.647 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2862.g61531d57.el 144/198 2024-04-17T06:07:58.672 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:19.0.0-2862.g61531d57.el9.x86_64 145/198 2024-04-17T06:07:58.696 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2862.g61531d57.el9.x86_64 145/198 2024-04-17T06:07:58.696 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:58.696 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-17T06:07:58.696 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:07:58.696 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-17T06:07:58.696 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:58.739 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:19.0.0-2862.g61531d57.el9.noarch 146/198 2024-04-17T06:07:58.751 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2862.g61531d57.el9.noarch 146/198 2024-04-17T06:07:58.791 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:19.0.0-2862.g61531d57.el9.x86_64 147/198 2024-04-17T06:07:59.665 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-17T06:07:59.691 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:19.0.0-2862.g61531d57.el9.x86_64 149/198 2024-04-17T06:07:59.715 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2862.g61531d57.el9.x86_64 149/198 2024-04-17T06:07:59.715 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:59.715 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-17T06:07:59.715 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:07:59.715 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-17T06:07:59.715 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:59.758 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2862.g61531 150/198 2024-04-17T06:07:59.781 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2862.g61531 150/198 2024-04-17T06:07:59.782 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:59.782 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-17T06:07:59.782 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:07:59.825 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 151/198 2024-04-17T06:07:59.849 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 151/198 2024-04-17T06:07:59.849 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:07:59.849 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-17T06:07:59.849 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:07:59.849 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-17T06:07:59.849 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:00.151 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 152/198 2024-04-17T06:08:00.175 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 152/198 2024-04-17T06:08:00.175 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-17T06:08:00.175 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-17T06:08:00.175 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:08:00.175 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-17T06:08:00.175 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:01.934 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:19.0.0-2862.g61531d57.el9.x86_64 1/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:19.0.0-2862.g61531d57.el9.x86_64 2/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 3/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 4/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2862.g61531 5/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:19.0.0-2862.g61531d57.el9.x86_64 6/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:19.0.0-2862.g61531d57.el9.x86_64 7/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:19.0.0-2862.g61531d57.el9.x86_64 8/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:19.0.0-2862.g61531d57.el9.x86_64 9/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:19.0.0-2862.g61531d57.el9.x86_64 10/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:19.0.0-2862.g61531d57.el9.x86_64 11/198 2024-04-17T06:08:01.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:19.0.0-2862.g61531d57.el9.x86_64 12/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 13/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-shell-2:19.0.0-2862.g61531d57.el9.x86_64 14/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:19.0.0-2862.g61531d57.el9.x86_ 15/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:19.0.0-2862.g61531d57.el9.x86_64 16/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:19.0.0-2862.g61531d57.el9.x86_64 17/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:19.0.0-2862.g61531d57.el9.x86_6 18/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:19.0.0-2862.g61531d57.el9.x86 19/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:19.0.0-2862.g61531d57.el9.x86_64 20/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2862.g61531d57.el 21/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:19.0.0-2862.g61531d57.el9. 22/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:19.0.0-2862.g61531d57.el9.x86_6 23/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:19.0.0-2862.g61531d57.el9.x86_64 24/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:19.0.0-2862.g61531d57.el9.x86_64 25/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:19.0.0-2862.g61531d57.el9.x86_64 26/198 2024-04-17T06:08:01.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 27/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 28/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:19.0.0-2862.g61531d57.el9.x86_64 29/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2862.g61531d57. 30/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2862.g61531d57.el9.noa 31/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2862.g61531d57.el9.n 32/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2862.g615 33/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2862.g61531d57.el 34/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2862.g61531d57.el9.noarch 35/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2862.g61531d57.e 36/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:19.0.0-2862.g61531d57.el9.noarch 37/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:19.0.0-2862.g61531d57.el9.noarch 38/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:19.0.0-2862.g61531d57.el9.noarch 39/198 2024-04-17T06:08:01.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T06:08:01.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T06:08:01.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T06:08:01.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T06:08:01.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T06:08:01.942 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T06:08:01.943 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T06:08:01.944 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T06:08:01.945 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:19.0.0-2862.g61531d57.el9.x86_64 151/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:19.0.0-2862.g61531d57.el9.x86_64 153/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T06:08:01.946 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T06:08:01.947 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:08:01.948 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T06:08:01.949 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T06:08:01.949 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T06:08:01.949 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:08:01.949 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T06:08:01.949 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T06:08:01.949 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T06:08:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:19.0.0-2862.g61531d57.el9.x86_64 153/198 2024-04-17T06:08:04.822 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 154/198 2024-04-17T06:08:04.853 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:19.0.0-2862.g61531d57.el9.x86_64 155/198 2024-04-17T06:08:04.879 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-17T06:08:05.061 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-shell-2:19.0.0-2862.g61531d57.el9.x86_64 157/198 2024-04-17T06:08:05.179 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-17T06:08:05.220 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 159/198 2024-04-17T06:08:05.254 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T06:08:05.254 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:05.254 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-04-17T06:08:05.254 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.255 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T06:08:05.256 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.257 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T06:08:05.258 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: jq-1.6-16.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T06:08:05.259 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T06:08:05.260 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T06:08:05.261 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T06:08:05.262 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T06:08:05.263 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T06:08:05.264 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T06:08:05.265 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.266 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:05.267 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-04-17T06:08:05.274 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-17T06:08:05.320 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-17T06:08:05.363 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-17T06:08:05.396 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:19.0.0-2862.g61531d57.el9.x86_ 163/198 2024-04-17T06:08:05.427 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-17T06:08:05.457 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-17T06:08:05.516 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-17T06:08:05.588 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-17T06:08:05.631 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-17T06:08:05.793 DEBUG:teuthology.parallel:result is None 2024-04-17T06:08:06.203 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:08:06.246 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:08:06.266 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-17T06:08:07.096 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:19.0.0-2862.g61531d57.el9.noarch 170/198 2024-04-17T06:08:07.177 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-17T06:08:07.217 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-17T06:08:07.299 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-17T06:08:07.330 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-17T06:08:07.355 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-17T06:08:07.380 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-17T06:08:07.407 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-17T06:08:07.407 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:08:07.495 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-17T06:08:07.525 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-17T06:08:07.525 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:08:07.610 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-17T06:08:07.636 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-17T06:08:07.669 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-17T06:08:07.694 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-17T06:08:07.719 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-17T06:08:07.755 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-17T06:08:07.801 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-17T06:08:07.832 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-17T06:08:07.857 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-17T06:08:07.883 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-17T06:08:07.908 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:08:07.934 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-17T06:08:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:08:07.941 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:08:08.128 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-17T06:08:08.128 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:08:08.219 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-17T06:08:08.251 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:08:08.278 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-17T06:08:08.304 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-17T06:08:08.328 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-17T06:08:08.395 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:08:13.949 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:19.0.0-2862.g61531d57.el9.x86_64 1/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:19.0.0-2862.g61531d57.el9.x86_64 2/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 3/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 4/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2862.g61531 5/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:19.0.0-2862.g61531d57.el9.x86_64 6/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:19.0.0-2862.g61531d57.el9.x86_64 7/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:19.0.0-2862.g61531d57.el9.x86_64 8/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:19.0.0-2862.g61531d57.el9.x86_64 9/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:19.0.0-2862.g61531d57.el9.x86_64 10/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:19.0.0-2862.g61531d57.el9.x86_64 11/198 2024-04-17T06:08:13.950 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:19.0.0-2862.g61531d57.el9.x86_64 12/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 13/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-shell-2:19.0.0-2862.g61531d57.el9.x86_64 14/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:19.0.0-2862.g61531d57.el9.x86_ 15/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:19.0.0-2862.g61531d57.el9.x86_64 16/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:19.0.0-2862.g61531d57.el9.x86_64 17/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:19.0.0-2862.g61531d57.el9.x86_6 18/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:19.0.0-2862.g61531d57.el9.x86 19/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:19.0.0-2862.g61531d57.el9.x86_64 20/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2862.g61531d57.el 21/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:19.0.0-2862.g61531d57.el9. 22/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:19.0.0-2862.g61531d57.el9.x86_6 23/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:19.0.0-2862.g61531d57.el9.x86_64 24/198 2024-04-17T06:08:13.951 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:19.0.0-2862.g61531d57.el9.x86_64 25/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:19.0.0-2862.g61531d57.el9.x86_64 26/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 27/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 28/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:19.0.0-2862.g61531d57.el9.x86_64 29/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2862.g61531d57. 30/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2862.g61531d57.el9.noa 31/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2862.g61531d57.el9.n 32/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2862.g615 33/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2862.g61531d57.el 34/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2862.g61531d57.el9.noarch 35/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2862.g61531d57.e 36/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:19.0.0-2862.g61531d57.el9.noarch 37/198 2024-04-17T06:08:13.952 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:19.0.0-2862.g61531d57.el9.noarch 38/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:19.0.0-2862.g61531d57.el9.noarch 39/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-17T06:08:13.953 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-17T06:08:13.954 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-17T06:08:13.955 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-17T06:08:13.956 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-17T06:08:13.957 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-17T06:08:13.958 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-17T06:08:13.959 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-17T06:08:13.960 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:19.0.0-2862.g61531d57.el9.x86_64 151/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:19.0.0-2862.g61531d57.el9.x86_64 153/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-17T06:08:13.961 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-17T06:08:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-17T06:08:13.963 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-17T06:08:13.964 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: acl-2.3.1-4.el9.x86_64 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-17T06:08:17.046 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.047 INFO:teuthology.orchestra.run.smithi190.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.048 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.049 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-shell-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:19.0.0-2862.g61531d57.el9.noarch 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: jq-1.6-16.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-17T06:08:17.050 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.051 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: lua-5.4.4-4.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.14-1.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-17T06:08:17.052 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-17T06:08:17.053 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-17T06:08:17.054 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2024-04-17T06:08:17.055 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.056 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-17T06:08:17.057 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:19.0.0-2862.g61531d57.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-3.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-17T06:08:17.058 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-17T06:08:17.059 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:17.059 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2024-04-17T06:08:17.533 DEBUG:teuthology.parallel:result is None 2024-04-17T06:08:17.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=61531d579f13bd3cc7ef7522484ee03eec1daad6 2024-04-17T06:08:17.752 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T06:08:17.779 INFO:teuthology.orchestra.run.smithi135.stdout:19.0.0-2862.g61531d57.el9 2024-04-17T06:08:17.780 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2862.g61531d57.el9 2024-04-17T06:08:17.780 INFO:teuthology.task.install:The correct ceph version 19.0.0-2862.g61531d57 is installed. 2024-04-17T06:08:17.782 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=61531d579f13bd3cc7ef7522484ee03eec1daad6 2024-04-17T06:08:17.921 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-17T06:08:17.945 INFO:teuthology.orchestra.run.smithi190.stdout:19.0.0-2862.g61531d57.el9 2024-04-17T06:08:17.945 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2862.g61531d57.el9 2024-04-17T06:08:17.945 INFO:teuthology.task.install:The correct ceph version 19.0.0-2862.g61531d57 is installed. 2024-04-17T06:08:17.947 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-17T06:08:17.947 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:17.947 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T06:08:17.983 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:17.983 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-17T06:08:18.017 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-17T06:08:18.017 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:18.017 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T06:08:18.056 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T06:08:18.126 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:18.126 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2024-04-17T06:08:18.156 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-17T06:08:18.224 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-17T06:08:18.224 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:18.224 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T06:08:18.254 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T06:08:18.323 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:18.323 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-17T06:08:18.352 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-17T06:08:18.419 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-17T06:08:18.420 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:18.420 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T06:08:18.448 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T06:08:18.517 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:18.517 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2024-04-17T06:08:18.545 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-17T06:08:18.614 INFO:teuthology.run_tasks:Running task ceph... 2024-04-17T06:08:18.700 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-17T06:08:18.700 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /var/log/ceph 2024-04-17T06:08:18.702 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2024-04-17T06:08:18.730 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-17T06:08:18.730 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T06:08:18.771 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-17T06:08:18.799 INFO:tasks.ceph:Creating extra log directories... 2024-04-17T06:08:18.799 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T06:08:18.839 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-17T06:08:18.873 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-17T06:08:18.874 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-17T06:08:18.874 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-17T06:08:18.874 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:18.874 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-17T06:08:18.920 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-17T06:08:19.043 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-17T06:08:19.071 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:19.071 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-17T06:08:19.086 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-17T06:08:19.208 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-17T06:08:19.242 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-17T06:08:19.242 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'missing required features', 'evicting unresponsive client'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-17T06:08:19.242 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi/7660092', 'branch': 'wip-jcollin-testing_11042024', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/admin}', 'email': 'jcollin@redhat.com', 'first_in_suite': False, 'job_id': '7660092', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'jcollin-2024-04-17_05:41:13-fs:functional-wip-jcollin-testing_11042024-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-jcollin-testing_11042024'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'missing required features', 'evicting unresponsive client'], 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'origin-jcollin-testing', 'sha1': '87915339f630682d277627759810c13965437320'}}, 'owner': 'scheduled_jcollin@teuthology', 'priority': 10, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8234, 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'origin-jcollin-testing', 'suite_path': '/home/teuthworker/src/github.com_joscollin_ceph_87915339f630682d277627759810c13965437320/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/joscollin/ceph.git', 'suite_sha1': '87915339f630682d277627759810c13965437320', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi135.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA+TbV4fUXchRFWNVx30oQ2h5DoTR8mbuXmPYha/GkR4nWTQhahSzXGnBZw8o7dN0NHl1w0jVB7dwyzUAHp1Lqg=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIQsBI5FskDhZ/+GwSSt32xHi5zYHljFY8J7yzNAJXZ/UOIPksg1MKaRckUMeDFKeOwi/1ZOM8XDlbzMQppBSX8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'lockdep': True, 'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'missing required features', 'evicting unresponsive client'], 'sha1': '61531d579f13bd3cc7ef7522484ee03eec1daad6', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6c637841c215537a4502385240412f1966e0faab', 'timestamp': '2024-04-17_05:41:13', 'tube': 'smithi', 'user': 'jcollin', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-17T06:08:19.243 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T06:08:19.245 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-17T06:08:19.292 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T06:08:19.300 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2024-04-17T06:08:19.360 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:19.360 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T06:08:19.381 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T06:08:19.381 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-17 06:07:56.654486990 +0000 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-17 06:06:02.409724347 +0000 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-17 06:06:02.409724347 +0000 2024-04-17T06:08:19.437 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-17 06:06:02.409724347 +0000 2024-04-17T06:08:19.437 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T06:08:19.504 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-17T06:08:19.504 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-17T06:08:19.504 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000273868 s, 1.9 MB/s 2024-04-17T06:08:19.506 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T06:08:19.562 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-04-17T06:08:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T06:08:19.618 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T06:08:19.618 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-17T06:08:19.618 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:08:19.618 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T06:08:19.618 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-17 06:07:56.655486969 +0000 2024-04-17T06:08:19.618 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-17 06:06:02.760717047 +0000 2024-04-17T06:08:19.618 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-17 06:06:02.760717047 +0000 2024-04-17T06:08:19.619 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-17 06:06:02.760717047 +0000 2024-04-17T06:08:19.619 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T06:08:19.685 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-17T06:08:19.685 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-17T06:08:19.685 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000186335 s, 2.7 MB/s 2024-04-17T06:08:19.687 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T06:08:19.742 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-17 06:07:56.655486969 +0000 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-17 06:06:03.101709955 +0000 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-17 06:06:03.101709955 +0000 2024-04-17T06:08:19.798 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-17 06:06:03.101709955 +0000 2024-04-17T06:08:19.799 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T06:08:19.864 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-17T06:08:19.864 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-17T06:08:19.865 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000259584 s, 2.0 MB/s 2024-04-17T06:08:19.865 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T06:08:19.922 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-04-17T06:08:19.977 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T06:08:19.977 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T06:08:19.977 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-04-17T06:08:19.977 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:08:19.977 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T06:08:19.977 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-04-17 06:07:56.656486948 +0000 2024-04-17T06:08:19.978 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-04-17 06:06:03.450702696 +0000 2024-04-17T06:08:19.978 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-04-17 06:06:03.450702696 +0000 2024-04-17T06:08:19.978 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-04-17 06:06:03.450702696 +0000 2024-04-17T06:08:19.978 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T06:08:20.045 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-04-17T06:08:20.045 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-04-17T06:08:20.045 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000206608 s, 2.5 MB/s 2024-04-17T06:08:20.046 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T06:08:20.103 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:20.103 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:20.103 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2024-04-17T06:08:20.121 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-17T06:08:20.122 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:08:08.757579508 +0000 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:06:02.398713232 +0000 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:06:02.398713232 +0000 2024-04-17T06:08:20.180 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:06:02.398713232 +0000 2024-04-17T06:08:20.181 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-17T06:08:20.249 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-04-17T06:08:20.249 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-04-17T06:08:20.249 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000217957 s, 2.3 MB/s 2024-04-17T06:08:20.251 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-17T06:08:20.309 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2024-04-17T06:08:20.366 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-17T06:08:20.367 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T06:08:20.367 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-17T06:08:20.367 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:08:20.367 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T06:08:20.367 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:08:08.758579479 +0000 2024-04-17T06:08:20.367 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:06:02.763702713 +0000 2024-04-17T06:08:20.367 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:06:02.763702713 +0000 2024-04-17T06:08:20.367 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:06:02.763702713 +0000 2024-04-17T06:08:20.367 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-17T06:08:20.437 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-04-17T06:08:20.437 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-04-17T06:08:20.437 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000217777 s, 2.4 MB/s 2024-04-17T06:08:20.438 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-17T06:08:20.497 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2024-04-17T06:08:20.553 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-17T06:08:20.553 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T06:08:20.553 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-17T06:08:20.553 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:08:20.553 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T06:08:20.554 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:08:08.758579479 +0000 2024-04-17T06:08:20.554 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:06:03.109692742 +0000 2024-04-17T06:08:20.554 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:06:03.109692742 +0000 2024-04-17T06:08:20.554 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:06:03.109692742 +0000 2024-04-17T06:08:20.554 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-17T06:08:20.621 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-04-17T06:08:20.621 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-04-17T06:08:20.621 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000193485 s, 2.6 MB/s 2024-04-17T06:08:20.623 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-17T06:08:20.680 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2024-04-17T06:08:20.735 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-17T06:08:20.736 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-17T06:08:20.736 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-17T06:08:20.736 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-17T06:08:20.736 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2024-04-17T06:08:20.736 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:08:08.758579479 +0000 2024-04-17T06:08:20.736 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:06:03.439683232 +0000 2024-04-17T06:08:20.736 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:06:03.439683232 +0000 2024-04-17T06:08:20.736 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:06:03.439683232 +0000 2024-04-17T06:08:20.736 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-17T06:08:20.802 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2024-04-17T06:08:20.803 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2024-04-17T06:08:20.803 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000157862 s, 3.2 MB/s 2024-04-17T06:08:20.804 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-17T06:08:20.860 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:20.860 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-17T06:08:20.860 INFO:tasks.ceph:Generating config... 2024-04-17T06:08:20.862 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-17T06:08:20.862 INFO:tasks.ceph:[client] debug client = 20 2024-04-17T06:08:20.862 INFO:tasks.ceph:[client] debug ms = 1 2024-04-17T06:08:20.862 INFO:tasks.ceph:[client] pid file = 2024-04-17T06:08:20.862 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-17T06:08:20.862 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-17T06:08:20.862 INFO:tasks.ceph:[global] lockdep = True 2024-04-17T06:08:20.862 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-17T06:08:20.863 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-17T06:08:20.864 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-17T06:08:20.864 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-17T06:08:20.864 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-17T06:08:20.864 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-17T06:08:20.864 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-17T06:08:20.864 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-17T06:08:20.864 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-17T06:08:20.864 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-17T06:08:20.865 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-17T06:08:20.866 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-17T06:08:20.866 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-17T06:08:20.866 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-17T06:08:20.866 INFO:tasks.ceph:Setting up mon.a... 2024-04-17T06:08:20.866 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-17T06:08:20.923 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.keyring 2024-04-17T06:08:20.926 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-17T06:08:20.975 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T06:08:21.003 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.135'), ('mon.b', '172.21.15.190'), ('mon.c', '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]')] 2024-04-17T06:08:21.004 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.135,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]', 'lockdep': True, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-17T06:08:21.005 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:21.005 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T06:08:21.059 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.135 --add b 172.21.15.190 --addv c '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:21.145 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: generated fsid 96728bf8-42ba-4aa8-88ba-489677bf3f41 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:setting min_mon_release = quincy 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:epoch 0 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:fsid 96728bf8-42ba-4aa8-88ba-489677bf3f41 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:last_changed 2024-04-17T06:08:21.148926+0000 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:created 2024-04-17T06:08:21.148926+0000 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:min_mon_release 17 (quincy) 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:election_strategy: 1 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:0: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.a 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:2: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.c 2024-04-17T06:08:21.146 INFO:teuthology.orchestra.run.smithi135.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-17T06:08:21.148 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-17T06:08:21.162 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 96728bf8-42ba-4aa8-88ba-489677bf3f41... 2024-04-17T06:08:21.163 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T06:08:21.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-17T06:08:21.255 INFO:teuthology.orchestra.run.smithi135.stdout:[global] 2024-04-17T06:08:21.255 INFO:teuthology.orchestra.run.smithi135.stdout: chdir = "" 2024-04-17T06:08:21.255 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: auth supported = cephx 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: filestore xattr use omap = true 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: mon clock drift allowed = 1.000 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: osd crush chooseleaf type = 0 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: auth debug = true 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on old message = true 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on bug = true 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.256 INFO:teuthology.orchestra.run.smithi135.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: mon pg warn max object skew = 0 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default size = 2 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow primary affinity = true 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd allow pg remap = true 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on legacy crush tunables = false 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on crush straw calc version zero = false 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on no sortbitwise = false 2024-04-17T06:08:21.257 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on osd down out interval zero = false 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: mon warn on too few osds = false 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: mon_allow_pool_size_one = true 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: osd default data pool replay window = 5 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: mon allow pool delete = true 2024-04-17T06:08:21.258 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: mon cluster log file level = debug 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: debug asserts on shutdown = true 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: mon health detail to clog = false 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: mon host = "172.21.15.135,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: lockdep = True 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: ms die on skipped message = False 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: fsid = 96728bf8-42ba-4aa8-88ba-489677bf3f41 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout:[osd] 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: osd journal size = 100 2024-04-17T06:08:21.259 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub load threshold = 5.0 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd scrub max interval = 600 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd mclock profile = high_recovery_ops 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd recover clone overlap = true 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd recovery max chunk = 1048576 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug shutdown = true 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug op order = true 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug verify stray on activate = true 2024-04-17T06:08:21.260 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug trim objects = true 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: osd open classes on start = true 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug pg log writeout = true 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: osd deep scrub update digest min age = 30 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: osd map max advance = 10 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: journal zero on create = true 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: filestore ondisk finisher threads = 3 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: filestore apply finisher threads = 3 2024-04-17T06:08:21.261 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: bdev debug aio = true 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: osd debug misdirected ops = true 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore block size = 96636764160 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: bluestore fsck on mount = True 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluefs = 20 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: debug bluestore = 20 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: debug osd = 20 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: debug rocksdb = 10 2024-04-17T06:08:21.262 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd full ratio = 0.9 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd nearfull ratio = 0.8 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: osd failsafe full ratio = 0.95 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: osd objectstore = bluestore 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr] 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: debug mgr = 20 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-04-17T06:08:21.263 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: mgr/telemetry/nag = false 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout:[mon] 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: debug mon = 20 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: debug paxos = 20 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: debug auth = 20 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: mon data avail warn = 5 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: mon mgr mkfs grace = 240 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min pgs per osd = 4 2024-04-17T06:08:21.264 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd reporter subtree level = osd 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: mon osd prime pg temp = true 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: mon reweight min bytes per osd = 10 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: # 1m isn't quite enough 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: mon_down_mkfs_grace = 2m 2024-04-17T06:08:21.265 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: mon_warn_on_filestore_osds = false 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: mon down mkfs grace = 300 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: mon op complaint time = 120 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout:[client] 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: rgw cache enabled = true 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable ops log = true 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: rgw enable usage log = true 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T06:08:21.266 INFO:teuthology.orchestra.run.smithi135.stdout: client mount timeout = 600 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout: debug client = 20 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout: pid file = "" 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.a] 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.b] 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout:[mon.c] 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout:[mds] 2024-04-17T06:08:21.267 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds = 20 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: debug mds balancer = 20 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: debug ms = 1 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug frag = True 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: mds debug scatterstat = True 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: mds op complaint time = 180 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: mds verify scatter = True 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: osd op complaint time = 180 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: rados mon op timeout = 900 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi135.stdout: rados osd op timeout = 900 2024-04-17T06:08:21.268 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2024-04-17T06:08:21.269 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2024-04-17T06:08:21.270 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.271 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.135,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: lockdep = True 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on skipped message = False 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 96728bf8-42ba-4aa8-88ba-489677bf3f41 2024-04-17T06:08:21.272 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2024-04-17T06:08:21.273 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug trim objects = true 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.274 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2024-04-17T06:08:21.275 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-04-17T06:08:21.276 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2024-04-17T06:08:21.277 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2024-04-17T06:08:21.278 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: mon down mkfs grace = 300 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2024-04-17T06:08:21.279 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2024-04-17T06:08:21.280 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2024-04-17T06:08:21.281 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2024-04-17T06:08:21.283 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-17T06:08:21.283 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-17T06:08:21.339 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-17T06:08:21.340 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:21.340 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-17T06:08:21.355 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:21.355 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-17T06:08:21.411 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:08:21.411 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:21.411 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T06:08:21.411 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T06:08:21.491 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:21.491 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:21.547 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T06:08:21.547 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:21.547 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-17T06:08:21.547 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-17T06:08:21.589 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:21.590 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:21.645 INFO:tasks.ceph:Setting up mon nodes... 2024-04-17T06:08:21.645 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-17T06:08:21.645 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-17T06:08:21.702 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-17T06:08:21.705 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T06:08:21.786 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-17T06:08:21.789 INFO:tasks.ceph:Setting up mds nodes... 2024-04-17T06:08:21.789 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-17T06:08:21.847 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-17T06:08:21.849 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-17T06:08:21.877 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-17T06:08:21.975 INFO:teuthology.orchestra.run.smithi135.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-17T06:08:21.977 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-17T06:08:22.005 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-17T06:08:22.063 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-17T06:08:22.065 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-17T06:08:22.093 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-17T06:08:22.190 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-17T06:08:22.193 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-17T06:08:22.221 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-17T06:08:22.221 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-17T06:08:22.265 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-17T06:08:22.281 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-17T06:08:22.324 INFO:teuthology.orchestra.run.smithi135.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-17T06:08:22.340 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-17T06:08:22.384 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-17T06:08:22.400 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-17T06:08:22.445 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-17T06:08:22.461 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-17T06:08:22.461 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi135.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-17T06:08:22.461 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-17T06:08:22.489 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:22.489 INFO:tasks.ceph:role: osd.0 2024-04-17T06:08:22.489 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:08:22.490 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T06:08:22.565 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T06:08:22.565 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T06:08:22.565 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T06:08:22.565 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-17T06:08:22.565 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T06:08:22.566 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-17T06:08:22.566 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T06:08:22.566 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-17T06:08:22.566 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T06:08:22.566 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T06:08:23.066 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-17T06:08:23.069 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-17T06:08:23.069 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-17T06:08:23.202 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-17T06:08:23.237 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-17T06:08:23.306 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:23.306 INFO:tasks.ceph:role: osd.1 2024-04-17T06:08:23.306 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:08:23.306 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T06:08:23.381 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T06:08:23.381 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T06:08:23.381 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T06:08:23.381 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-17T06:08:23.381 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T06:08:23.382 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-17T06:08:23.382 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T06:08:23.382 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-17T06:08:23.382 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T06:08:23.382 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T06:08:23.845 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-17T06:08:23.846 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-17T06:08:23.847 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-17T06:08:23.940 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-17T06:08:23.975 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-17T06:08:24.043 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:24.043 INFO:tasks.ceph:role: osd.2 2024-04-17T06:08:24.043 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:08:24.043 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T06:08:24.119 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T06:08:24.119 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T06:08:24.119 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T06:08:24.119 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-17T06:08:24.119 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T06:08:24.119 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-17T06:08:24.119 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T06:08:24.119 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-17T06:08:24.120 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T06:08:24.120 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T06:08:24.604 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-17T06:08:24.606 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-17T06:08:24.606 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-17T06:08:24.718 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-17T06:08:24.753 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-17T06:08:24.821 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:24.821 INFO:tasks.ceph:role: osd.3 2024-04-17T06:08:24.821 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:08:24.821 DEBUG:teuthology.orchestra.run.smithi135:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T06:08:24.897 INFO:teuthology.orchestra.run.smithi135.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout: = sunit=0 swidth=0 blks 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T06:08:24.898 INFO:teuthology.orchestra.run.smithi135.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T06:08:25.412 INFO:teuthology.orchestra.run.smithi135.stdout:Discarding blocks...Done. 2024-04-17T06:08:25.414 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi135.front.sepia.ceph.com -o noatime 2024-04-17T06:08:25.414 DEBUG:teuthology.orchestra.run.smithi135:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-17T06:08:25.539 DEBUG:teuthology.orchestra.run.smithi135:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-17T06:08:25.577 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:25.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:25.678+0000 7f69d9d1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-17T06:08:25.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:25.679+0000 7f69d9d1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-17T06:08:25.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:25.679+0000 7f69d9d1b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T06:08:25.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:25.679+0000 7f69d9d1b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T06:08:25.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:25.679+0000 7f69d9d1b640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-17T06:08:25.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:25.679+0000 7f69d9d1b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-17T06:08:28.531 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-17T06:08:28.564 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:28.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:28.659+0000 7fc19020d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-17T06:08:28.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:28.659+0000 7fc19020d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-17T06:08:28.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:28.659+0000 7fc19020d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T06:08:28.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:28.659+0000 7fc19020d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T06:08:28.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:28.659+0000 7fc19020d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-17T06:08:28.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:28.660+0000 7fc19020d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-17T06:08:31.507 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-17T06:08:31.540 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:31.637+0000 7f3c81d47640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-17T06:08:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:31.637+0000 7f3c81d47640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-17T06:08:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:31.637+0000 7f3c81d47640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T06:08:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:31.637+0000 7f3c81d47640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T06:08:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:31.637+0000 7f3c81d47640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-17T06:08:31.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:31.638+0000 7f3c81d47640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-17T06:08:34.492 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-17T06:08:34.525 DEBUG:teuthology.orchestra.run.smithi135:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:34.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:34.622+0000 7fd7014b5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-17T06:08:34.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:34.622+0000 7fd7014b5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-17T06:08:34.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:34.622+0000 7fd7014b5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T06:08:34.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:34.622+0000 7fd7014b5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T06:08:34.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:34.622+0000 7fd7014b5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-17T06:08:34.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:34.623+0000 7fd7014b5640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-17T06:08:37.474 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-17T06:08:37.507 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-17T06:08:37.542 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:37.542 INFO:tasks.ceph:role: osd.4 2024-04-17T06:08:37.543 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T06:08:37.543 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-17T06:08:37.621 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T06:08:37.621 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T06:08:37.622 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T06:08:37.622 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-17T06:08:37.622 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T06:08:37.622 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-04-17T06:08:37.622 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T06:08:37.622 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-17T06:08:37.622 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T06:08:37.622 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T06:08:38.135 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-04-17T06:08:38.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-04-17T06:08:38.138 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-17T06:08:38.269 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-17T06:08:38.307 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-17T06:08:38.376 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:38.376 INFO:tasks.ceph:role: osd.5 2024-04-17T06:08:38.376 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T06:08:38.377 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T06:08:38.453 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T06:08:38.915 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-04-17T06:08:38.917 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-04-17T06:08:38.917 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-17T06:08:39.002 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-17T06:08:39.074 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-17T06:08:39.143 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:39.143 INFO:tasks.ceph:role: osd.6 2024-04-17T06:08:39.143 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T06:08:39.143 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-17T06:08:39.219 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T06:08:39.219 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T06:08:39.220 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T06:08:39.220 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-17T06:08:39.220 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T06:08:39.220 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-04-17T06:08:39.220 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T06:08:39.220 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-17T06:08:39.220 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T06:08:39.220 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T06:08:39.718 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-04-17T06:08:39.720 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-04-17T06:08:39.720 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-17T06:08:39.875 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-17T06:08:39.913 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-17T06:08:39.983 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-17T06:08:39.983 INFO:tasks.ceph:role: osd.7 2024-04-17T06:08:39.983 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T06:08:39.984 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-17T06:08:40.062 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-17T06:08:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-17T06:08:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-17T06:08:40.062 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-17T06:08:40.063 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-17T06:08:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2024-04-17T06:08:40.063 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-17T06:08:40.063 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-17T06:08:40.063 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-17T06:08:40.063 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-17T06:08:40.558 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2024-04-17T06:08:40.560 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2024-04-17T06:08:40.560 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-17T06:08:40.674 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-17T06:08:40.712 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:40.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:40.823+0000 7fb1b618a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-17T06:08:40.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:40.823+0000 7fb1b618a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-17T06:08:40.821 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:40.823+0000 7fb1b618a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T06:08:40.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:40.823+0000 7fb1b618a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T06:08:40.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:40.823+0000 7fb1b618a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-17T06:08:40.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:40.824+0000 7fb1b618a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-17T06:08:43.668 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-17T06:08:43.702 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:43.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:43.797+0000 7f8e67a19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-17T06:08:43.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:43.797+0000 7f8e67a19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-17T06:08:43.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:43.797+0000 7f8e67a19640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T06:08:43.795 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:43.797+0000 7f8e67a19640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T06:08:43.796 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:43.797+0000 7f8e67a19640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-17T06:08:43.796 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:43.798+0000 7f8e67a19640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-17T06:08:46.643 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-17T06:08:46.676 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:46.769 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:46.771+0000 7f6eb220f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-17T06:08:46.769 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:46.771+0000 7f6eb220f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-17T06:08:46.769 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:46.771+0000 7f6eb220f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T06:08:46.770 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:46.771+0000 7f6eb220f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T06:08:46.770 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:46.771+0000 7f6eb220f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-17T06:08:46.770 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:46.772+0000 7f6eb220f640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-17T06:08:49.243 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:08:49.246 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:08:49.622 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-17T06:08:49.652 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:49.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:49.743+0000 7f54f0689640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-17T06:08:49.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:49.744+0000 7f54f0689640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-17T06:08:49.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:49.744+0000 7f54f0689640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T06:08:49.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:49.744+0000 7f54f0689640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T06:08:49.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:49.744+0000 7f54f0689640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-17T06:08:49.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:49.744+0000 7f54f0689640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-17T06:08:52.594 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-17T06:08:52.628 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-17T06:08:52.628 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:52.628 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-17T06:08:52.661 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:52.662 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-17T06:08:52.734 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:52.734 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-17T06:08:52.807 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:52.807 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-17T06:08:52.879 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:52.879 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-17T06:08:52.950 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:52.951 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-17T06:08:53.021 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:53.021 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-17T06:08:53.092 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.092 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-17T06:08:53.126 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.126 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-17T06:08:53.197 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.197 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-17T06:08:53.268 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.268 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-17T06:08:53.337 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.337 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-17T06:08:53.407 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.407 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-17T06:08:53.476 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.476 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-17T06:08:53.544 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:53.544 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-17T06:08:53.563 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:53.564 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-17T06:08:53.622 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.622 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-17T06:08:53.638 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:53.638 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-17T06:08:53.694 INFO:tasks.ceph:Adding keys to all mons... 2024-04-17T06:08:53.695 DEBUG:teuthology.orchestra.run.smithi135:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T06:08:53.697 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-17T06:08:53.737 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.y] 2024-04-17T06:08:53.737 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBVZx9mag0UKhAAZErsmKPrmQnltFsGEY3G0Q== 2024-04-17T06:08:53.737 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.a] 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBVZx9mwV6rMhAA+B8sT5k/vUBR/B97UZtNYg== 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.c] 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBVZx9mdERUOhAASUbdm3i3s/yjZtUy8BnSnQ== 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.0] 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBZZx9m2rGCKBAAbzZVYc+eHBEsfL6fCbslQg== 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.1] 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBcZx9mRW5aJxAAzSSCL8AxDpsuTK3DVBC+Cg== 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.2] 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBfZx9m2wwJJhAAtxdZwlcjcDoAloZT4/2FDw== 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.3] 2024-04-17T06:08:53.738 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBiZx9m5x0oJRAAG93B3QVvH2tREQsjFDdzQA== 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:[mgr.x] 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBVZx9mKqoXLxAAgdNLvJfyvFLBM4ZtNb44PQ== 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.b] 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBWZx9m1+31AxAAAF2amMrczR6a53kgr6jvSg== 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:[mds.d] 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBWZx9mIxuRCxAAu+KJ6Y1OKIdqK2gcpGYPOg== 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.4] 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBoZx9mThkiMRAAA5EpOIEBihnj6sfsuU1+/Q== 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.5] 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBrZx9m//aSLxAAmQz7rOpw28uMG6aojXJf7g== 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.6] 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBuZx9mLHoFLhAAON/rLUW0eaUbghEPtcoKTw== 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:[osd.7] 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBxZx9mBxBjLBAAkv3QB5M4G6vn7+CXhpfqCQ== 2024-04-17T06:08:53.739 INFO:teuthology.orchestra.run.smithi135.stdout:[client.0] 2024-04-17T06:08:53.740 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBWZx9mgVoAEBAAlX2QvDOCLGvYpV4AcvC0GQ== 2024-04-17T06:08:53.740 INFO:teuthology.orchestra.run.smithi135.stdout:[client.2] 2024-04-17T06:08:53.740 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBWZx9mXZ6IExAA8kTDkMxm2R3pKSsR9sv90g== 2024-04-17T06:08:53.740 INFO:teuthology.orchestra.run.smithi135.stdout:[client.1] 2024-04-17T06:08:53.740 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBWZx9mBg0fFxAAl/vQU1o5OV8YYnz+IQu+oA== 2024-04-17T06:08:53.740 INFO:teuthology.orchestra.run.smithi135.stdout:[client.3] 2024-04-17T06:08:53.740 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBWZx9mb8q6GhAAHgO6FH2ZBUnmOYdECsUF2Q== 2024-04-17T06:08:53.760 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.y] 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBVZx9mag0UKhAAZErsmKPrmQnltFsGEY3G0Q== 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBVZx9mwV6rMhAA+B8sT5k/vUBR/B97UZtNYg== 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBVZx9mdERUOhAASUbdm3i3s/yjZtUy8BnSnQ== 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBZZx9m2rGCKBAAbzZVYc+eHBEsfL6fCbslQg== 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBcZx9mRW5aJxAAzSSCL8AxDpsuTK3DVBC+Cg== 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBfZx9m2wwJJhAAtxdZwlcjcDoAloZT4/2FDw== 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2024-04-17T06:08:53.761 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBiZx9m5x0oJRAAG93B3QVvH2tREQsjFDdzQA== 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBVZx9mKqoXLxAAgdNLvJfyvFLBM4ZtNb44PQ== 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBWZx9m1+31AxAAAF2amMrczR6a53kgr6jvSg== 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBWZx9mIxuRCxAAu+KJ6Y1OKIdqK2gcpGYPOg== 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBoZx9mThkiMRAAA5EpOIEBihnj6sfsuU1+/Q== 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBrZx9m//aSLxAAmQz7rOpw28uMG6aojXJf7g== 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.6] 2024-04-17T06:08:53.762 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBuZx9mLHoFLhAAON/rLUW0eaUbghEPtcoKTw== 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.7] 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBxZx9mBxBjLBAAkv3QB5M4G6vn7+CXhpfqCQ== 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBWZx9mgVoAEBAAlX2QvDOCLGvYpV4AcvC0GQ== 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout:[client.2] 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBWZx9mXZ6IExAA8kTDkMxm2R3pKSsR9sv90g== 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBWZx9mBg0fFxAAl/vQU1o5OV8YYnz+IQu+oA== 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout:[client.3] 2024-04-17T06:08:53.763 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQBWZx9mb8q6GhAAHgO6FH2ZBUnmOYdECsUF2Q== 2024-04-17T06:08:53.764 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T06:08:53.783 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T06:08:53.858 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T06:08:53.861 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T06:08:53.918 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T06:08:53.921 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T06:08:53.977 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:53.980 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.034 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.036 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.090 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.093 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.147 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.149 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.203 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T06:08:54.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-17T06:08:54.260 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T06:08:54.263 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T06:08:54.319 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T06:08:54.323 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-17T06:08:54.379 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.382 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.435 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.438 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.492 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.494 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.549 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.551 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-17T06:08:54.605 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T06:08:54.607 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T06:08:54.664 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T06:08:54.666 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T06:08:54.724 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T06:08:54.726 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T06:08:54.782 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T06:08:54.785 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-17T06:08:54.841 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-17T06:08:54.842 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-17T06:08:54.870 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T06:08:54.969 DEBUG:teuthology.orchestra.run.smithi135:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-17T06:08:54.997 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-17T06:08:55.026 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T06:08:55.125 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-17T06:08:55.153 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-17T06:08:55.222 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-17T06:08:55.322 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-17T06:08:55.350 DEBUG:teuthology.orchestra.run.smithi135:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:55.353 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-17T06:08:55.406 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-17T06:08:55.406 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-17T06:08:55.406 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-17T06:08:55.409 INFO:tasks.ceph.mon.a:Started 2024-04-17T06:08:55.409 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-17T06:08:55.409 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-17T06:08:55.448 INFO:tasks.ceph.mon.b:Started 2024-04-17T06:08:55.449 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-17T06:08:55.449 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-17T06:08:55.451 INFO:tasks.ceph.mon.c:Started 2024-04-17T06:08:55.451 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-17T06:08:55.451 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-17T06:08:55.451 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-17T06:08:55.454 INFO:tasks.ceph.mgr.y:Started 2024-04-17T06:08:55.454 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-17T06:08:55.454 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-17T06:08:55.457 INFO:tasks.ceph.mgr.x:Started 2024-04-17T06:08:55.457 INFO:tasks.ceph:Setting crush tunables to default 2024-04-17T06:08:55.457 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --cluster ceph osd crush tunables default 2024-04-17T06:08:55.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.563+0000 7f29a3a63640 1 Processor -- start 2024-04-17T06:08:55.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.563+0000 7f29a3a63640 1 -- start start 2024-04-17T06:08:55.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a3a63640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 0x7f299c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:55.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a3a63640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f299c06caa0 0x7f299c09af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a3a63640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f299c0cd4a0 con 0x7f299c0d0ea0 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a3a63640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f299c0d1940 con 0x7f299c06caa0 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a3a63640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f299c0b1d80 con 0x7f299c0b19a0 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a0ffe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f299c0d0ea0 0x7f299c0cdf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57278/0 (socket says 172.21.15.135:57278) 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a0ffe640 1 -- 172.21.15.135:0/344619211 learned_addr learned my addr 172.21.15.135:0/344619211 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a17ff640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 0x7f299c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a2000640 1 --2- 172.21.15.135:0/344619211 >> v2:172.21.15.190:3300/0 conn(0x7f299c06caa0 0x7f299c09af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a17ff640 1 -- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 msgr2=0x7f299c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a17ff640 1 -- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 msgr2=0x7f299c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a2000640 1 -- 172.21.15.135:0/344619211 >> v2:172.21.15.190:3300/0 conn(0x7f299c06caa0 msgr2=0x7f299c09af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a17ff640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 0x7f299c09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a2000640 1 -- 172.21.15.135:0/344619211 >> v2:172.21.15.190:3300/0 conn(0x7f299c06caa0 msgr2=0x7f299c09af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a2000640 1 --2- 172.21.15.135:0/344619211 >> v2:172.21.15.190:3300/0 conn(0x7f299c06caa0 0x7f299c09af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-17T06:08:55.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a17ff640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 0x7f299c09a9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T06:08:55.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.564+0000 7f29a2000640 1 --2- 172.21.15.135:0/344619211 >> v2:172.21.15.190:3300/0 conn(0x7f299c06caa0 0x7f299c09af30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-17T06:08:55.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1549145942 0 0) 0x7f299c0cd4a0 con 0x7f299c0d0ea0 2024-04-17T06:08:55.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2984003520 con 0x7f299c0d0ea0 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3779712510 0 0) 0x7f2984003520 con 0x7f299c0d0ea0 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 msgr2=0x7f299c09a9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 0x7f299c09a9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 >> v2:172.21.15.190:3300/0 conn(0x7f299c06caa0 msgr2=0x7f299c09af30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 --2- 172.21.15.135:0/344619211 >> v2:172.21.15.190:3300/0 conn(0x7f299c06caa0 0x7f299c09af30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c0d1380 con 0x7f299c0d0ea0 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f2990001670 con 0x7f299c0d0ea0 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 >> v1:172.21.15.135:6789/0 conn(0x7f299c0d0ea0 legacy=0x7f299c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29840039b0 0x7f2984003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2984004800 0x7f2984004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 0x7f29840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:55.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0d1940 con 0x7f2984004800 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0b1d80 con 0x7f29840039b0 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f298400ba70 con 0x7f299c0b19a0 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f29a0ffe640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2984004800 0x7f2984004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f29a2000640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 0x7f29840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f29a17ff640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29840039b0 0x7f2984003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f29a0ffe640 1 -- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 msgr2=0x7f29840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f29a0ffe640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f299c0b19a0 0x7f29840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f29a0ffe640 1 -- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29840039b0 msgr2=0x7f2984003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f29a0ffe640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29840039b0 0x7f2984003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.728+0000 7f29a0ffe640 1 -- 172.21.15.135:0/344619211 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2984003520 con 0x7f2984004800 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.729+0000 7f29a0ffe640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2984004800 0x7f2984004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2990001eb0 tx=0x7f2984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3742bba321fc97 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.729+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990007d70 con 0x7f2984004800 2024-04-17T06:08:55.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.729+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2990030070 con 0x7f2984004800 2024-04-17T06:08:55.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.729+0000 7f299bfff640 1 -- 172.21.15.135:0/344619211 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990005d30 con 0x7f2984004800 2024-04-17T06:08:55.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.730+0000 7f29a3a63640 1 -- 172.21.15.135:0/344619211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2984004800 msgr2=0x7f2984004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:55.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.730+0000 7f29a3a63640 1 --2- 172.21.15.135:0/344619211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2984004800 0x7f2984004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2990001eb0 tx=0x7f2984005830 comp rx=0 tx=0).stop 2024-04-17T06:08:55.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.730+0000 7f29a2000640 1 -- 172.21.15.135:0/344619211 reap_dead start 2024-04-17T06:08:55.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.730+0000 7f29a3a63640 1 -- 172.21.15.135:0/344619211 shutdown_connections 2024-04-17T06:08:55.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.730+0000 7f29a3a63640 1 -- 172.21.15.135:0/344619211 >> 172.21.15.135:0/344619211 conn(0x7f299c0b41a0 msgr2=0x7f299c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:55.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.730+0000 7f29a3a63640 1 -- 172.21.15.135:0/344619211 shutdown_connections 2024-04-17T06:08:55.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.730+0000 7f29a3a63640 1 -- 172.21.15.135:0/344619211 wait complete. 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.731+0000 7f29a3a63640 1 Processor -- start 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.731+0000 7f29a3a63640 1 -- start start 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a3a63640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2984004800 0x7f299c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a3a63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f299c154f40 0x7f299c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a3a63640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0cb3b0 0x7f299c0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a3a63640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0cda20 con 0x7f299c154f40 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a3a63640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0cd740 con 0x7f299c0cb3b0 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a3a63640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f299c09a9f0 con 0x7f2984004800 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a0ffe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f299c154f40 0x7f299c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a0ffe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f299c154f40 0x7f299c154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54450/0 (socket says 172.21.15.135:54450) 2024-04-17T06:08:55.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a17ff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2984004800 0x7f299c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a0ffe640 1 -- 172.21.15.135:0/2971641990 learned_addr learned my addr 172.21.15.135:0/2971641990 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a2000640 1 --2- 172.21.15.135:0/2971641990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0cb3b0 0x7f299c0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a0ffe640 1 -- 172.21.15.135:0/2971641990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2984004800 msgr2=0x7f299c156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a0ffe640 1 --2- 172.21.15.135:0/2971641990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2984004800 0x7f299c156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a0ffe640 1 -- 172.21.15.135:0/2971641990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0cb3b0 msgr2=0x7f299c0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a0ffe640 1 --2- 172.21.15.135:0/2971641990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0cb3b0 0x7f299c0b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.732+0000 7f29a0ffe640 1 -- 172.21.15.135:0/2971641990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c154780 con 0x7f299c154f40 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.733+0000 7f29a0ffe640 1 --2- 172.21.15.135:0/2971641990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f299c154f40 0x7f299c154240 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f299002c540 tx=0x7f2990000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.733+0000 7f2999ffb640 1 -- 172.21.15.135:0/2971641990 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990045070 con 0x7f299c154f40 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.733+0000 7f2999ffb640 1 -- 172.21.15.135:0/2971641990 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2990004bf0 con 0x7f299c154f40 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.733+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c0cbbe0 con 0x7f299c154f40 2024-04-17T06:08:55.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.733+0000 7f2999ffb640 1 -- 172.21.15.135:0/2971641990 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990007d50 con 0x7f299c154f40 2024-04-17T06:08:55.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.733+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c0b56e0 con 0x7f299c154f40 2024-04-17T06:08:55.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.733+0000 7f2999ffb640 1 -- 172.21.15.135:0/2971641990 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2990007070 con 0x7f299c154f40 2024-04-17T06:08:55.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.733+0000 7f2999ffb640 1 -- 172.21.15.135:0/2971641990 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2990035af0 con 0x7f299c154f40 2024-04-17T06:08:55.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.734+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f299c06e680 con 0x7f299c154f40 2024-04-17T06:08:55.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.736+0000 7f2999ffb640 1 -- 172.21.15.135:0/2971641990 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f299c06e680 con 0x7f299c154f40 2024-04-17T06:08:55.802 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.804+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f299c068a20 con 0x7f299c154f40 2024-04-17T06:08:55.803 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.805+0000 7f2999ffb640 1 -- 172.21.15.135:0/2971641990 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f2990035d40 con 0x7f299c154f40 2024-04-17T06:08:55.803 INFO:teuthology.orchestra.run.smithi135.stderr:adjusted tunables profile to default 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f299c154f40 msgr2=0x7f299c154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 --2- 172.21.15.135:0/2971641990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f299c154f40 0x7f299c154240 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f299002c540 tx=0x7f2990000fe0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 shutdown_connections 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 --2- 172.21.15.135:0/2971641990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f299c0cb3b0 0x7f299c0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 --2- 172.21.15.135:0/2971641990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f299c154f40 0x7f299c154240 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 --2- 172.21.15.135:0/2971641990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2984004800 0x7f299c156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 >> 172.21.15.135:0/2971641990 conn(0x7f299c0b41a0 msgr2=0x7f299c0af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 shutdown_connections 2024-04-17T06:08:55.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:55.806+0000 7f29a3a63640 1 -- 172.21.15.135:0/2971641990 wait complete. 2024-04-17T06:08:55.815 INFO:tasks.ceph:check_enable_crimson: False 2024-04-17T06:08:55.815 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-17T06:08:55.815 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:55.816 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-17T06:08:55.844 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:55.844 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-17T06:08:55.912 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:55.912 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-17T06:08:55.980 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:08:55.981 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-17T06:08:56.049 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:56.049 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-17T06:08:56.079 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:56.079 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-17T06:08:56.149 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:56.149 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-17T06:08:56.191 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:56.194+0000 7f67df814040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T06:08:56.217 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:08:56.217 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-17T06:08:56.240 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:56.242+0000 7f6d2441a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-17T06:08:56.249 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:56.251+0000 7f67df814040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T06:08:56.276 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 4df49209-004c-4834-b05c-d4681c7e0f19 0 2024-04-17T06:08:56.293 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:56.295+0000 7f6d2441a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-17T06:08:56.420 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.422+0000 7f0953022640 1 Processor -- start 2024-04-17T06:08:56.420 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.422+0000 7f0953022640 1 -- start start 2024-04-17T06:08:56.422 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.423+0000 7f0953022640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f094c15ebd0 0x7f094c070200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.422 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.423+0000 7f0953022640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f094c0b3ea0 0x7f094c06c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.422 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.423+0000 7f0953022640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f094c0b47d0 con 0x7f094c06cee0 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.423+0000 7f0953022640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f094c073a20 con 0x7f094c15ebd0 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.423+0000 7f0953022640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f094c0b4a70 con 0x7f094c0b3ea0 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.423+0000 7f094bfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f094c0b3ea0 0x7f094c06c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.423+0000 7f094bfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f094c0b3ea0 0x7f094c06c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45030/0 (socket says 172.21.15.190:45030) 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.423+0000 7f094bfff640 1 -- 172.21.15.190:0/3237405316 learned_addr learned my addr 172.21.15.190:0/3237405316 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.424+0000 7f0950dbe640 1 --2- 172.21.15.190:0/3237405316 >> v2:172.21.15.135:3300/0 conn(0x7f094c15ebd0 0x7f094c070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.424+0000 7f094bfff640 1 -- 172.21.15.190:0/3237405316 >> v2:172.21.15.135:3300/0 conn(0x7f094c15ebd0 msgr2=0x7f094c070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.424+0000 7f094bfff640 1 --2- 172.21.15.190:0/3237405316 >> v2:172.21.15.135:3300/0 conn(0x7f094c15ebd0 0x7f094c070200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.424+0000 7f094bfff640 1 -- 172.21.15.190:0/3237405316 >> v1:172.21.15.135:6789/0 conn(0x7f094c06cee0 legacy=0x7f094c06e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.424+0000 7f094bfff640 1 -- 172.21.15.190:0/3237405316 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c073460 con 0x7f094c0b3ea0 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.424+0000 7f0950dbe640 1 --2- 172.21.15.190:0/3237405316 >> v2:172.21.15.135:3300/0 conn(0x7f094c15ebd0 0x7f094c070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.424+0000 7f094bfff640 1 --2- 172.21.15.190:0/3237405316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c0b3ea0 0x7f094c06c9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f094c09f080 tx=0x7f093802ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1acf882af2abba95 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f094b7fe640 1 -- 172.21.15.190:0/3237405316 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093802f850 con 0x7f094c0b3ea0 2024-04-17T06:08:56.424 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f094b7fe640 1 -- 172.21.15.190:0/3237405316 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093802f9f0 con 0x7f094c0b3ea0 2024-04-17T06:08:56.424 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f094b7fe640 1 -- 172.21.15.190:0/3237405316 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093802fdc0 con 0x7f094c0b3ea0 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f0953022640 1 -- 172.21.15.190:0/3237405316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c0b3ea0 msgr2=0x7f094c06c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f0953022640 1 --2- 172.21.15.190:0/3237405316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c0b3ea0 0x7f094c06c9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f094c09f080 tx=0x7f093802ba30 comp rx=0 tx=0).stop 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f0953022640 1 -- 172.21.15.190:0/3237405316 shutdown_connections 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f0953022640 1 --2- 172.21.15.190:0/3237405316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c0b3ea0 0x7f094c06c9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f0953022640 1 --2- 172.21.15.190:0/3237405316 >> v2:172.21.15.135:3300/0 conn(0x7f094c15ebd0 0x7f094c070200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f0953022640 1 -- 172.21.15.190:0/3237405316 >> 172.21.15.190:0/3237405316 conn(0x7f094c072780 msgr2=0x7f094c072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f0953022640 1 -- 172.21.15.190:0/3237405316 shutdown_connections 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.425+0000 7f0953022640 1 -- 172.21.15.190:0/3237405316 wait complete. 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.426+0000 7f0953022640 1 Processor -- start 2024-04-17T06:08:56.427 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.426+0000 7f0953022640 1 -- start start 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0953022640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c06cee0 0x7f094c1f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0953022640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094c0b3ea0 0x7f094c15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0953022640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f094c15ebd0 0x7f094c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0953022640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f094c15efb0 con 0x7f094c15ebd0 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0953022640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f094c0b4d50 con 0x7f094c06cee0 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0953022640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f094c15f130 con 0x7f094c0b3ea0 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0950dbe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c06cee0 0x7f094c1f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f094bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094c0b3ea0 0x7f094c15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0950dbe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c06cee0 0x7f094c1f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45038/0 (socket says 172.21.15.190:45038) 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f0950dbe640 1 -- 172.21.15.190:0/1203524614 learned_addr learned my addr 172.21.15.190:0/1203524614 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.427+0000 7f09515bf640 1 --2- 172.21.15.190:0/1203524614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f094c15ebd0 0x7f094c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f0950dbe640 1 -- 172.21.15.190:0/1203524614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094c0b3ea0 msgr2=0x7f094c15c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.428 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f0950dbe640 1 --2- 172.21.15.190:0/1203524614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094c0b3ea0 0x7f094c15c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f0950dbe640 1 -- 172.21.15.190:0/1203524614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f094c15ebd0 msgr2=0x7f094c15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f0950dbe640 1 --2- 172.21.15.190:0/1203524614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f094c15ebd0 0x7f094c15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f0950dbe640 1 -- 172.21.15.190:0/1203524614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c158730 con 0x7f094c06cee0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f0950dbe640 1 --2- 172.21.15.190:0/1203524614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c06cee0 0x7f094c1f4bb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f094000d7d0 tx=0x7f094000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f09497fa640 1 -- 172.21.15.190:0/1203524614 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0940004030 con 0x7f094c06cee0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f09497fa640 1 -- 172.21.15.190:0/1203524614 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09400041d0 con 0x7f094c06cee0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f09497fa640 1 -- 172.21.15.190:0/1203524614 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09400096a0 con 0x7f094c06cee0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094c158a10 con 0x7f094c06cee0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094c158fd0 con 0x7f094c06cee0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f09497fa640 1 -- 172.21.15.190:0/1203524614 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0940009840 con 0x7f094c06cee0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.428+0000 7f09497fa640 1 -- 172.21.15.190:0/1203524614 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f0940014770 con 0x7f094c06cee0 2024-04-17T06:08:56.429 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.429+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094c068a20 con 0x7f094c06cee0 2024-04-17T06:08:56.430 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.431+0000 7f09497fa640 1 -- 172.21.15.190:0/1203524614 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0940009c70 con 0x7f094c06cee0 2024-04-17T06:08:56.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.503+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4df49209-004c-4834-b05c-d4681c7e0f19", "id": 0} v 0) -- 0x7f094c0b0360 con 0x7f094c06cee0 2024-04-17T06:08:56.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.506+0000 7f09497fa640 1 -- 172.21.15.190:0/1203524614 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4df49209-004c-4834-b05c-d4681c7e0f19", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f0940003610 con 0x7f094c06cee0 2024-04-17T06:08:56.505 INFO:teuthology.orchestra.run.smithi190.stdout:0 2024-04-17T06:08:56.507 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c06cee0 msgr2=0x7f094c1f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.507 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 --2- 172.21.15.190:0/1203524614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c06cee0 0x7f094c1f4bb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f094000d7d0 tx=0x7f094000dc90 comp rx=0 tx=0).stop 2024-04-17T06:08:56.507 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 shutdown_connections 2024-04-17T06:08:56.507 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 --2- 172.21.15.190:0/1203524614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f094c15ebd0 0x7f094c15c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.508 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 --2- 172.21.15.190:0/1203524614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f094c0b3ea0 0x7f094c15c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.508 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 --2- 172.21.15.190:0/1203524614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f094c06cee0 0x7f094c1f4bb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.508 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 >> 172.21.15.190:0/1203524614 conn(0x7f094c072780 msgr2=0x7f094c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.508 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 shutdown_connections 2024-04-17T06:08:56.508 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.507+0000 7f0953022640 1 -- 172.21.15.190:0/1203524614 wait complete. 2024-04-17T06:08:56.516 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new a4181f82-fde6-4d13-96fb-3ddbf31a53ae 1 2024-04-17T06:08:56.619 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.621+0000 7fb8517f2640 1 Processor -- start 2024-04-17T06:08:56.619 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.621+0000 7fb8517f2640 1 -- start start 2024-04-17T06:08:56.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb8517f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb8517f2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84c0b51b0 con 0x7fb84c0cfed0 2024-04-17T06:08:56.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb8517f2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84c0d0420 con 0x7fb84c070880 2024-04-17T06:08:56.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb8517f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb84c0d0700 con 0x7fb84c15b9f0 2024-04-17T06:08:56.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb84affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb84b7fe640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fb84c070880 0x7fb84c0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:49592/0 (socket says 172.21.15.190:49592) 2024-04-17T06:08:56.622 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb84affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c0b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:36670/0 (socket says 172.21.15.190:36670) 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb84b7fe640 1 -- 172.21.15.190:0/1796732383 learned_addr learned my addr 172.21.15.190:0/1796732383 (peer_addr_for_me v1:172.21.15.190:0/0) 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb84affd640 1 -- 172.21.15.190:0/1796732383 >> v1:172.21.15.190:6789/0 conn(0x7fb84c070880 legacy=0x7fb84c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb84affd640 1 -- 172.21.15.190:0/1796732383 >> v1:172.21.15.135:6789/0 conn(0x7fb84c0cfed0 legacy=0x7fb84c073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb84affd640 1 -- 172.21.15.190:0/1796732383 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c0cf780 con 0x7fb84c15b9f0 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.622+0000 7fb849ffb640 1 -- 172.21.15.190:0/1796732383 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 822123816 0 0) 0x7fb84c0d0420 con 0x7fb84c070880 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.623+0000 7fb84affd640 1 --2- 172.21.15.190:0/1796732383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c0b4c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0097d0 tx=0x7fb83c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c75380d55beef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.623+0000 7fb849ffb640 1 -- 172.21.15.190:0/1796732383 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c035020 con 0x7fb84c15b9f0 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.623+0000 7fb849ffb640 1 -- 172.21.15.190:0/1796732383 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83c031800 con 0x7fb84c15b9f0 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.623+0000 7fb849ffb640 1 -- 172.21.15.190:0/1796732383 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c031bd0 con 0x7fb84c15b9f0 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.624+0000 7fb8517f2640 1 -- 172.21.15.190:0/1796732383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 msgr2=0x7fb84c0b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.624+0000 7fb8517f2640 1 --2- 172.21.15.190:0/1796732383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c0b4c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0097d0 tx=0x7fb83c02b8a0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.623 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.624+0000 7fb8517f2640 1 -- 172.21.15.190:0/1796732383 shutdown_connections 2024-04-17T06:08:56.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.624+0000 7fb8517f2640 1 --2- 172.21.15.190:0/1796732383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c0b4c70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.624+0000 7fb8517f2640 1 -- 172.21.15.190:0/1796732383 >> 172.21.15.190:0/1796732383 conn(0x7fb84c0b3320 msgr2=0x7fb84c0b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.624+0000 7fb8517f2640 1 -- 172.21.15.190:0/1796732383 shutdown_connections 2024-04-17T06:08:56.624 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.624+0000 7fb8517f2640 1 -- 172.21.15.190:0/1796732383 wait complete. 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.625+0000 7fb8517f2640 1 Processor -- start 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.625+0000 7fb8517f2640 1 -- start start 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb8517f2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb84c070880 0x7fb84c1f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb8517f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c0cfed0 0x7fb84c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb8517f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb8517f2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c070ce0 con 0x7fb84c070880 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb8517f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c0cf530 con 0x7fb84c0cfed0 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb8517f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb84c070e60 con 0x7fb84c15b9f0 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c15d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:36680/0 (socket says 172.21.15.190:36680) 2024-04-17T06:08:56.627 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84a7fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c0cfed0 0x7fb84c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84b7fe640 1 -- 172.21.15.190:0/1762837137 learned_addr learned my addr 172.21.15.190:0/1762837137 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84affd640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb84c070880 0x7fb84c1f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84b7fe640 1 -- 172.21.15.190:0/1762837137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c0cfed0 msgr2=0x7fb84c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84b7fe640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c0cfed0 0x7fb84c15d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84b7fe640 1 -- 172.21.15.190:0/1762837137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb84c070880 msgr2=0x7fb84c1f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84b7fe640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb84c070880 0x7fb84c1f5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84b7fe640 1 -- 172.21.15.190:0/1762837137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c1f26b0 con 0x7fb84c15b9f0 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.626+0000 7fb84a7fc640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c0cfed0 0x7fb84c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.627+0000 7fb84b7fe640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c15d900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c7a0 tx=0x7fb84400cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.627+0000 7fb8437fe640 1 -- 172.21.15.190:0/1762837137 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844018070 con 0x7fb84c15b9f0 2024-04-17T06:08:56.628 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.627+0000 7fb8437fe640 1 -- 172.21.15.190:0/1762837137 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84400a640 con 0x7fb84c15b9f0 2024-04-17T06:08:56.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.627+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c1f2990 con 0x7fb84c15b9f0 2024-04-17T06:08:56.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.627+0000 7fb8437fe640 1 -- 172.21.15.190:0/1762837137 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84400aab0 con 0x7fb84c15b9f0 2024-04-17T06:08:56.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.627+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c1f2f50 con 0x7fb84c15b9f0 2024-04-17T06:08:56.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.627+0000 7fb8437fe640 1 -- 172.21.15.190:0/1762837137 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb84401b070 con 0x7fb84c15b9f0 2024-04-17T06:08:56.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.627+0000 7fb8437fe640 1 -- 172.21.15.190:0/1762837137 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fb84401d7f0 con 0x7fb84c15b9f0 2024-04-17T06:08:56.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.628+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84c068a20 con 0x7fb84c15b9f0 2024-04-17T06:08:56.629 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.630+0000 7fb8437fe640 1 -- 172.21.15.190:0/1762837137 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb8440078f0 con 0x7fb84c15b9f0 2024-04-17T06:08:56.718 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.720+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a4181f82-fde6-4d13-96fb-3ddbf31a53ae", "id": 1} v 0) -- 0x7fb84c0ceb60 con 0x7fb84c15b9f0 2024-04-17T06:08:56.729 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.731+0000 7fb8437fe640 1 -- 172.21.15.190:0/1762837137 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4181f82-fde6-4d13-96fb-3ddbf31a53ae", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fb84400ad20 con 0x7fb84c15b9f0 2024-04-17T06:08:56.729 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 msgr2=0x7fb84c15d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c15d900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c7a0 tx=0x7fb84400cc60 comp rx=0 tx=0).stop 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 shutdown_connections 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb84c15b9f0 0x7fb84c15d900 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb84c0cfed0 0x7fb84c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 --2- 172.21.15.190:0/1762837137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb84c070880 0x7fb84c1f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 >> 172.21.15.190:0/1762837137 conn(0x7fb84c0b3320 msgr2=0x7fb84c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 shutdown_connections 2024-04-17T06:08:56.731 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.732+0000 7fb8517f2640 1 -- 172.21.15.190:0/1762837137 wait complete. 2024-04-17T06:08:56.741 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 250dc157-79ef-4459-906e-5ccc3b85ce94 2 2024-04-17T06:08:56.837 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:56.839+0000 7f67df814040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T06:08:56.837 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:56.840+0000 7f6d2441a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-17T06:08:56.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.847+0000 7f87de042640 1 Processor -- start 2024-04-17T06:08:56.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.847+0000 7f87de042640 1 -- start start 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.847+0000 7f87de042640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80cef20 0x7f87d8160460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.847+0000 7f87de042640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d80d0510 con 0x7f87d81609a0 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.847+0000 7f87de042640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d80cf470 con 0x7f87d8070f90 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.847+0000 7f87de042640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f87d8160e10 con 0x7f87d80cef20 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87cffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80cef20 0x7f87d8160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87cffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80cef20 0x7f87d8160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:36684/0 (socket says 172.21.15.190:36684) 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87d77fe640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f87d8070f90 0x7f87d80aeb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:49604/0 (socket says 172.21.15.190:49604) 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87cffff640 1 -- 172.21.15.190:0/744401475 learned_addr learned my addr 172.21.15.190:0/744401475 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87d6ffd640 1 -- 172.21.15.190:0/744401475 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884267183 0 0) 0x7f87d80cf470 con 0x7f87d8070f90 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87d6ffd640 1 -- 172.21.15.190:0/744401475 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87c4003520 con 0x7f87d8070f90 2024-04-17T06:08:56.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87cffff640 1 -- 172.21.15.190:0/744401475 >> v1:172.21.15.190:6789/0 conn(0x7f87d8070f90 legacy=0x7f87d80aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87cffff640 1 -- 172.21.15.190:0/744401475 >> v1:172.21.15.135:6789/0 conn(0x7f87d81609a0 legacy=0x7f87d8152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.848+0000 7f87cffff640 1 -- 172.21.15.190:0/744401475 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d80cf750 con 0x7f87d80cef20 2024-04-17T06:08:56.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.849+0000 7f87cffff640 1 --2- 172.21.15.190:0/744401475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80cef20 0x7f87d8160460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87c00098a0 tx=0x7f87c002ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b037fe3b97511ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.849+0000 7f87d6ffd640 1 -- 172.21.15.190:0/744401475 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c002f830 con 0x7f87d80cef20 2024-04-17T06:08:56.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.849+0000 7f87d6ffd640 1 -- 172.21.15.190:0/744401475 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c002f9d0 con 0x7f87d80cef20 2024-04-17T06:08:56.848 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.849+0000 7f87d6ffd640 1 -- 172.21.15.190:0/744401475 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c002fda0 con 0x7f87d80cef20 2024-04-17T06:08:56.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.850+0000 7f87de042640 1 -- 172.21.15.190:0/744401475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80cef20 msgr2=0x7f87d8160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.850+0000 7f87de042640 1 --2- 172.21.15.190:0/744401475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80cef20 0x7f87d8160460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87c00098a0 tx=0x7f87c002ba10 comp rx=0 tx=0).stop 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.850+0000 7f87de042640 1 -- 172.21.15.190:0/744401475 shutdown_connections 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.850+0000 7f87de042640 1 --2- 172.21.15.190:0/744401475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80cef20 0x7f87d8160460 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.850+0000 7f87de042640 1 -- 172.21.15.190:0/744401475 >> 172.21.15.190:0/744401475 conn(0x7f87d80cd7b0 msgr2=0x7f87d80cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.850+0000 7f87de042640 1 -- 172.21.15.190:0/744401475 shutdown_connections 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.850+0000 7f87de042640 1 -- 172.21.15.190:0/744401475 wait complete. 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.851+0000 7f87de042640 1 Processor -- start 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.851+0000 7f87de042640 1 -- start start 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87de042640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d8070f90 0x7f87d81443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87de042640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87d80cef20 0x7f87d8153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87de042640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d81609a0 0x7f87d80ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:56.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87de042640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87d80cfd10 con 0x7f87d80cef20 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87de042640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f87d80d0080 con 0x7f87d8070f90 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87de042640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f87d806c9a0 con 0x7f87d81609a0 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d8070f90 0x7f87d81443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d81609a0 0x7f87d80ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d8070f90 0x7f87d81443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45062/0 (socket says 172.21.15.190:45062) 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 -- 172.21.15.190:0/103453805 learned_addr learned my addr 172.21.15.190:0/103453805 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87cffff640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87d80cef20 0x7f87d8153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 -- 172.21.15.190:0/103453805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d81609a0 msgr2=0x7f87d80ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d81609a0 0x7f87d80ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 -- 172.21.15.190:0/103453805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87d80cef20 msgr2=0x7f87d8153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87d80cef20 0x7f87d8153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 -- 172.21.15.190:0/103453805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d8146e50 con 0x7f87d8070f90 2024-04-17T06:08:56.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d7fff640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d81609a0 0x7f87d80ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d77fe640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d8070f90 0x7f87d81443a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c500 tx=0x7f87c800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d4ff9640 1 -- 172.21.15.190:0/103453805 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8018070 con 0x7f87d8070f90 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d4ff9640 1 -- 172.21.15.190:0/103453805 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c8007500 con 0x7f87d8070f90 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d8147130 con 0x7f87d8070f90 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.852+0000 7f87d4ff9640 1 -- 172.21.15.190:0/103453805 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8004a20 con 0x7f87d8070f90 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.853+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d81476f0 con 0x7f87d8070f90 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.853+0000 7f87d4ff9640 1 -- 172.21.15.190:0/103453805 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87c801b070 con 0x7f87d8070f90 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.853+0000 7f87d4ff9640 1 -- 172.21.15.190:0/103453805 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f87c8012860 con 0x7f87d8070f90 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.853+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d8068a20 con 0x7f87d8070f90 2024-04-17T06:08:56.857 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.855+0000 7f87d4ff9640 1 -- 172.21.15.190:0/103453805 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f87c8004c80 con 0x7f87d8070f90 2024-04-17T06:08:56.891 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:56.893+0000 7f6d2441a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T06:08:56.894 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:56.896+0000 7f67df814040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-17T06:08:56.925 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.927+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "250dc157-79ef-4459-906e-5ccc3b85ce94", "id": 2} v 0) -- 0x7f87d80ced00 con 0x7f87d8070f90 2024-04-17T06:08:56.930 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.932+0000 7f87d4ff9640 1 -- 172.21.15.190:0/103453805 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "250dc157-79ef-4459-906e-5ccc3b85ce94", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f87c8012510 con 0x7f87d8070f90 2024-04-17T06:08:56.930 INFO:teuthology.orchestra.run.smithi190.stdout:2 2024-04-17T06:08:56.931 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d8070f90 msgr2=0x7f87d81443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:56.931 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d8070f90 0x7f87d81443a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c500 tx=0x7f87c800c9c0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.931 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 shutdown_connections 2024-04-17T06:08:56.931 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d81609a0 0x7f87d80ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.931 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87d80cef20 0x7f87d8153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.931 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 --2- 172.21.15.190:0/103453805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d8070f90 0x7f87d81443a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:56.931 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 >> 172.21.15.190:0/103453805 conn(0x7f87d80cd7b0 msgr2=0x7f87d80ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:56.931 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 shutdown_connections 2024-04-17T06:08:56.932 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:56.933+0000 7f87de042640 1 -- 172.21.15.190:0/103453805 wait complete. 2024-04-17T06:08:56.937 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:56.939+0000 7f6d2441a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T06:08:56.941 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:56.943+0000 7f67df814040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-17T06:08:56.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 5719faec-5aaf-4944-87d0-aafafb5107bd 3 2024-04-17T06:08:56.991 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:56.993+0000 7f6d2441a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T06:08:56.998 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.000+0000 7f67df814040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-17T06:08:57.033 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.035+0000 7f6d2441a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T06:08:57.042 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.044+0000 7f67df814040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-17T06:08:57.044 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.046+0000 7fc449b3d640 1 Processor -- start 2024-04-17T06:08:57.044 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.046+0000 7fc449b3d640 1 -- start start 2024-04-17T06:08:57.046 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc449b3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44415c930 0x7fc44409b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.046 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc449b3d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc44409b100 0x7fc4440b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc449b3d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4440cd440 con 0x7fc4440d0490 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc449b3d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc4440c7210 con 0x7fc44409b100 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc449b3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc4440cd720 con 0x7fc44415c930 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44415c930 0x7fc44409b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44415c930 0x7fc44409b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:36706/0 (socket says 172.21.15.190:36706) 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc4437fe640 1 -- 172.21.15.190:0/2819640342 learned_addr learned my addr 172.21.15.190:0/2819640342 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.047+0000 7fc443fff640 1 --2- 172.21.15.190:0/2819640342 >> v2:172.21.15.135:3300/0 conn(0x7fc44409b100 0x7fc4440b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.048+0000 7fc4437fe640 1 -- 172.21.15.190:0/2819640342 >> v2:172.21.15.135:3300/0 conn(0x7fc44409b100 msgr2=0x7fc4440b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.048+0000 7fc4437fe640 1 --2- 172.21.15.190:0/2819640342 >> v2:172.21.15.135:3300/0 conn(0x7fc44409b100 0x7fc4440b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.048+0000 7fc4437fe640 1 -- 172.21.15.190:0/2819640342 >> v1:172.21.15.135:6789/0 conn(0x7fc4440d0490 legacy=0x7fc44409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.048+0000 7fc4437fe640 1 -- 172.21.15.190:0/2819640342 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440cda00 con 0x7fc44415c930 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.048+0000 7fc4437fe640 1 --2- 172.21.15.190:0/2819640342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44415c930 0x7fc44409b8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc434009870 tx=0x7fc43402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d5ee082aa6a459 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc442ffd640 1 -- 172.21.15.190:0/2819640342 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434035020 con 0x7fc44415c930 2024-04-17T06:08:57.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc442ffd640 1 -- 172.21.15.190:0/2819640342 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc434031640 con 0x7fc44415c930 2024-04-17T06:08:57.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc442ffd640 1 -- 172.21.15.190:0/2819640342 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43402d040 con 0x7fc44415c930 2024-04-17T06:08:57.051 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc449b3d640 1 -- 172.21.15.190:0/2819640342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44415c930 msgr2=0x7fc44409b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc449b3d640 1 --2- 172.21.15.190:0/2819640342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44415c930 0x7fc44409b8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc434009870 tx=0x7fc43402b8f0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc449b3d640 1 -- 172.21.15.190:0/2819640342 shutdown_connections 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc449b3d640 1 --2- 172.21.15.190:0/2819640342 >> v2:172.21.15.135:3300/0 conn(0x7fc44409b100 0x7fc4440b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc449b3d640 1 --2- 172.21.15.190:0/2819640342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44415c930 0x7fc44409b8d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc449b3d640 1 -- 172.21.15.190:0/2819640342 >> 172.21.15.190:0/2819640342 conn(0x7fc4440b41a0 msgr2=0x7fc4440b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc449b3d640 1 -- 172.21.15.190:0/2819640342 shutdown_connections 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.049+0000 7fc449b3d640 1 -- 172.21.15.190:0/2819640342 wait complete. 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.050+0000 7fc449b3d640 1 Processor -- start 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc449b3d640 1 -- start start 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc449b3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44409b100 0x7fc444156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.052 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc449b3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440d0490 0x7fc4441f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc449b3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc44415c930 0x7fc4440b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc449b3d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc44409be10 con 0x7fc44415c930 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc449b3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440cdfc0 con 0x7fc4440d0490 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc449b3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc4440ce4e0 con 0x7fc44409b100 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc43affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440d0490 0x7fc4441f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.051+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44409b100 0x7fc444156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc43affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440d0490 0x7fc4441f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45074/0 (socket says 172.21.15.190:45074) 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc43affd640 1 -- 172.21.15.190:0/3344473317 learned_addr learned my addr 172.21.15.190:0/3344473317 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc443fff640 1 --2- 172.21.15.190:0/3344473317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc44415c930 0x7fc4440b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc43affd640 1 -- 172.21.15.190:0/3344473317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44409b100 msgr2=0x7fc444156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc43affd640 1 --2- 172.21.15.190:0/3344473317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44409b100 0x7fc444156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc43affd640 1 -- 172.21.15.190:0/3344473317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc44415c930 msgr2=0x7fc4440b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.053 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc43affd640 1 --2- 172.21.15.190:0/3344473317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc44415c930 0x7fc4440b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc43affd640 1 -- 172.21.15.190:0/3344473317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4441f4cd0 con 0x7fc4440d0490 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc43affd640 1 --2- 172.21.15.190:0/3344473317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440d0490 0x7fc4441f4790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00c3c0 tx=0x7fc42c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc440ff9640 1 -- 172.21.15.190:0/3344473317 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0052d0 con 0x7fc4440d0490 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc440ff9640 1 -- 172.21.15.190:0/3344473317 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c005490 con 0x7fc4440d0490 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc440ff9640 1 -- 172.21.15.190:0/3344473317 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c007260 con 0x7fc4440d0490 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4440b6490 con 0x7fc4440d0490 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4440b6a50 con 0x7fc4440d0490 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc440ff9640 1 -- 172.21.15.190:0/3344473317 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc42c007400 con 0x7fc4440d0490 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.052+0000 7fc440ff9640 1 -- 172.21.15.190:0/3344473317 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fc42c0067d0 con 0x7fc4440d0490 2024-04-17T06:08:57.054 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.053+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc444068a20 con 0x7fc4440d0490 2024-04-17T06:08:57.055 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.055+0000 7fc440ff9640 1 -- 172.21.15.190:0/3344473317 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc42c0035f0 con 0x7fc4440d0490 2024-04-17T06:08:57.124 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.126+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5719faec-5aaf-4944-87d0-aafafb5107bd", "id": 3} v 0) -- 0x7fc4440b5040 con 0x7fc4440d0490 2024-04-17T06:08:57.127 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.130+0000 7fc440ff9640 1 -- 172.21.15.190:0/3344473317 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5719faec-5aaf-4944-87d0-aafafb5107bd", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fc42c006af0 con 0x7fc4440d0490 2024-04-17T06:08:57.128 INFO:teuthology.orchestra.run.smithi190.stdout:3 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440d0490 msgr2=0x7fc4441f4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 --2- 172.21.15.190:0/3344473317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440d0490 0x7fc4441f4790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00c3c0 tx=0x7fc42c00c880 comp rx=0 tx=0).stop 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 shutdown_connections 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 --2- 172.21.15.190:0/3344473317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc44415c930 0x7fc4440b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 --2- 172.21.15.190:0/3344473317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4440d0490 0x7fc4441f4790 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 --2- 172.21.15.190:0/3344473317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc44409b100 0x7fc444156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 >> 172.21.15.190:0/3344473317 conn(0x7fc4440b41a0 msgr2=0x7fc4440d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 shutdown_connections 2024-04-17T06:08:57.129 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.131+0000 7fc449b3d640 1 -- 172.21.15.190:0/3344473317 wait complete. 2024-04-17T06:08:57.139 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 970a8be4-0bb2-46ad-8e4e-127c3d58b2f7 4 2024-04-17T06:08:57.151 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.153+0000 7f6d2441a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T06:08:57.167 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.169+0000 7f67df814040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-17T06:08:57.220 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.222+0000 7f6d2441a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T06:08:57.239 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.241+0000 7f67df814040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-17T06:08:57.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.245+0000 7fd496f40640 1 Processor -- start 2024-04-17T06:08:57.242 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.245+0000 7fd496f40640 1 -- start start 2024-04-17T06:08:57.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.245+0000 7fd496f40640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4900d0490 0x7fd49009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd496f40640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4900cd440 con 0x7fd49009af30 2024-04-17T06:08:57.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd496f40640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4900cd720 con 0x7fd49015c930 2024-04-17T06:08:57.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd496f40640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd4900c7210 con 0x7fd4900d0490 2024-04-17T06:08:57.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd48ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4900d0490 0x7fd49009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.244 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd48ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4900d0490 0x7fd49009a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45090/0 (socket says 172.21.15.190:45090) 2024-04-17T06:08:57.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd48ffff640 1 -- 172.21.15.190:0/3642257018 learned_addr learned my addr 172.21.15.190:0/3642257018 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd48ffff640 1 -- 172.21.15.190:0/3642257018 >> v1:172.21.15.190:6789/0 conn(0x7fd49015c930 legacy=0x7fd49009b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd48ffff640 1 -- 172.21.15.190:0/3642257018 >> v1:172.21.15.135:6789/0 conn(0x7fd49009af30 legacy=0x7fd4900b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.245 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.246+0000 7fd48ffff640 1 -- 172.21.15.190:0/3642257018 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4900cda00 con 0x7fd4900d0490 2024-04-17T06:08:57.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd48ffff640 1 --2- 172.21.15.190:0/3642257018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4900d0490 0x7fd49009a9f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009870 tx=0x7fd47c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f913d7fb2750b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd48f7fe640 1 -- 172.21.15.190:0/3642257018 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c034070 con 0x7fd4900d0490 2024-04-17T06:08:57.246 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd48f7fe640 1 -- 172.21.15.190:0/3642257018 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c030860 con 0x7fd4900d0490 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd48f7fe640 1 -- 172.21.15.190:0/3642257018 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c030c30 con 0x7fd4900d0490 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd496f40640 1 -- 172.21.15.190:0/3642257018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4900d0490 msgr2=0x7fd49009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd496f40640 1 --2- 172.21.15.190:0/3642257018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4900d0490 0x7fd49009a9f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009870 tx=0x7fd47c02b9a0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd496f40640 1 -- 172.21.15.190:0/3642257018 shutdown_connections 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd496f40640 1 --2- 172.21.15.190:0/3642257018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4900d0490 0x7fd49009a9f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd496f40640 1 -- 172.21.15.190:0/3642257018 >> 172.21.15.190:0/3642257018 conn(0x7fd4900b41a0 msgr2=0x7fd4900b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.247+0000 7fd496f40640 1 -- 172.21.15.190:0/3642257018 shutdown_connections 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.248+0000 7fd496f40640 1 -- 172.21.15.190:0/3642257018 wait complete. 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.249+0000 7fd496f40640 1 Processor -- start 2024-04-17T06:08:57.247 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.249+0000 7fd496f40640 1 -- start start 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.249+0000 7fd496f40640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49009af30 0x7fd490155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.249+0000 7fd496f40640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4900d0490 0x7fd49006dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.249+0000 7fd496f40640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd49015c930 0x7fd4900b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd496f40640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4900cdfc0 con 0x7fd49009af30 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd496f40640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd49009b360 con 0x7fd49015c930 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd496f40640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4900ce4e0 con 0x7fd4900d0490 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd49015c930 0x7fd4900b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd48ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4900d0490 0x7fd49006dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd49015c930 0x7fd4900b79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45094/0 (socket says 172.21.15.190:45094) 2024-04-17T06:08:57.250 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 -- 172.21.15.190:0/673823855 learned_addr learned my addr 172.21.15.190:0/673823855 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd494cdc640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49009af30 0x7fd490155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 -- 172.21.15.190:0/673823855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4900d0490 msgr2=0x7fd49006dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4900d0490 0x7fd49006dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 -- 172.21.15.190:0/673823855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49009af30 msgr2=0x7fd490155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49009af30 0x7fd490155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 -- 172.21.15.190:0/673823855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4900b5fc0 con 0x7fd49015c930 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd48ffff640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4900d0490 0x7fd49006dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd4954dd640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd49015c930 0x7fd4900b79f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd48800c6b0 tx=0x7fd48800cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd48d7fa640 1 -- 172.21.15.190:0/673823855 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488003b70 con 0x7fd49015c930 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd48d7fa640 1 -- 172.21.15.190:0/673823855 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd488003d30 con 0x7fd49015c930 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4900b62a0 con 0x7fd49015c930 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd48d7fa640 1 -- 172.21.15.190:0/673823855 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488005310 con 0x7fd49015c930 2024-04-17T06:08:57.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.250+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4900b6860 con 0x7fd49015c930 2024-04-17T06:08:57.252 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.251+0000 7fd48d7fa640 1 -- 172.21.15.190:0/673823855 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4880054b0 con 0x7fd49015c930 2024-04-17T06:08:57.252 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.251+0000 7fd48d7fa640 1 -- 172.21.15.190:0/673823855 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fd4880129d0 con 0x7fd49015c930 2024-04-17T06:08:57.252 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.252+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd490068a20 con 0x7fd49015c930 2024-04-17T06:08:57.252 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.254+0000 7fd48d7fa640 1 -- 172.21.15.190:0/673823855 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd488012c70 con 0x7fd49015c930 2024-04-17T06:08:57.265 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.268+0000 7f6d2441a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T06:08:57.298 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.300+0000 7f67df814040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-17T06:08:57.317 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.319+0000 7f6d2441a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T06:08:57.336 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.339+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "970a8be4-0bb2-46ad-8e4e-127c3d58b2f7", "id": 4} v 0) -- 0x7fd490069230 con 0x7fd49015c930 2024-04-17T06:08:57.342 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.344+0000 7fd48d7fa640 1 -- 172.21.15.190:0/673823855 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "970a8be4-0bb2-46ad-8e4e-127c3d58b2f7", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fd4880124c0 con 0x7fd49015c930 2024-04-17T06:08:57.342 INFO:teuthology.orchestra.run.smithi190.stdout:4 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd49015c930 msgr2=0x7fd4900b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd49015c930 0x7fd4900b79f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd48800c6b0 tx=0x7fd48800cb70 comp rx=0 tx=0).stop 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 shutdown_connections 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd49015c930 0x7fd4900b79f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4900d0490 0x7fd49006dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 --2- 172.21.15.190:0/673823855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd49009af30 0x7fd490155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 >> 172.21.15.190:0/673823855 conn(0x7fd4900b41a0 msgr2=0x7fd4900b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 shutdown_connections 2024-04-17T06:08:57.344 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.345+0000 7fd496f40640 1 -- 172.21.15.190:0/673823855 wait complete. 2024-04-17T06:08:57.352 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.355+0000 7f67df814040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-17T06:08:57.354 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 79cd6222-7c39-4b17-bf01-253339bd1678 5 2024-04-17T06:08:57.364 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.367+0000 7f6d2441a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T06:08:57.401 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.403+0000 7f67df814040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-17T06:08:57.497 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.499+0000 7fe8e3f24640 1 Processor -- start 2024-04-17T06:08:57.497 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.499+0000 7fe8e3f24640 1 -- start start 2024-04-17T06:08:57.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e3f24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe8dc09b100 0x7fe8dc06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e3f24640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8dc06d4b0 0x7fe8dc0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e3f24640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8dc15f110 con 0x7fe8dc15ed30 2024-04-17T06:08:57.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e3f24640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe8dc0ccd00 con 0x7fe8dc09b100 2024-04-17T06:08:57.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e3f24640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe8dc0cd2c0 con 0x7fe8dc06d4b0 2024-04-17T06:08:57.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e24c1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8dc06d4b0 0x7fe8dc0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.499 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e24c1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe8dc06d4b0 0x7fe8dc0cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45102/0 (socket says 172.21.15.190:45102) 2024-04-17T06:08:57.499 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e24c1640 1 -- 172.21.15.190:0/1265408760 learned_addr learned my addr 172.21.15.190:0/1265408760 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.499 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e14bf640 1 --2- 172.21.15.190:0/1265408760 >> v2:172.21.15.135:3300/0 conn(0x7fe8dc09b100 0x7fe8dc06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.499 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e24c1640 1 -- 172.21.15.190:0/1265408760 >> v2:172.21.15.135:3300/0 conn(0x7fe8dc09b100 msgr2=0x7fe8dc06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:57.499 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e24c1640 1 --2- 172.21.15.190:0/1265408760 >> v2:172.21.15.135:3300/0 conn(0x7fe8dc09b100 0x7fe8dc06ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.499 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e24c1640 1 -- 172.21.15.190:0/1265408760 >> v1:172.21.15.190:6789/0 conn(0x7fe8dc15ed30 legacy=0x7fe8dc09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.499 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.500+0000 7fe8e24c1640 1 -- 172.21.15.190:0/1265408760 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc06dbe0 con 0x7fe8dc06d4b0 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.501+0000 7fe8e24c1640 1 --2- 172.21.15.190:0/1265408760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc06d4b0 0x7fe8dc0cd750 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc073cb0 tx=0x7fe8d402bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c154d9865095de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.501+0000 7fe8e0cbe640 1 -- 172.21.15.190:0/1265408760 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d402ca80 con 0x7fe8dc06d4b0 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.501+0000 7fe8e0cbe640 1 -- 172.21.15.190:0/1265408760 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d402cc20 con 0x7fe8dc06d4b0 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.501+0000 7fe8e0cbe640 1 -- 172.21.15.190:0/1265408760 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40343f0 con 0x7fe8dc06d4b0 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.502+0000 7fe8e3f24640 1 -- 172.21.15.190:0/1265408760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc06d4b0 msgr2=0x7fe8dc0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.502+0000 7fe8e3f24640 1 --2- 172.21.15.190:0/1265408760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc06d4b0 0x7fe8dc0cd750 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc073cb0 tx=0x7fe8d402bee0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.502+0000 7fe8e3f24640 1 -- 172.21.15.190:0/1265408760 shutdown_connections 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.502+0000 7fe8e3f24640 1 --2- 172.21.15.190:0/1265408760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc06d4b0 0x7fe8dc0cd750 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.502+0000 7fe8e3f24640 1 --2- 172.21.15.190:0/1265408760 >> v2:172.21.15.135:3300/0 conn(0x7fe8dc09b100 0x7fe8dc06ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.502+0000 7fe8e3f24640 1 -- 172.21.15.190:0/1265408760 >> 172.21.15.190:0/1265408760 conn(0x7fe8dc0b41a0 msgr2=0x7fe8dc0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.502+0000 7fe8e3f24640 1 -- 172.21.15.190:0/1265408760 shutdown_connections 2024-04-17T06:08:57.501 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.502+0000 7fe8e3f24640 1 -- 172.21.15.190:0/1265408760 wait complete. 2024-04-17T06:08:57.502 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.503+0000 7fe8e3f24640 1 Processor -- start 2024-04-17T06:08:57.502 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.503+0000 7fe8e3f24640 1 -- start start 2024-04-17T06:08:57.504 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e3f24640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8dc06d4b0 0x7fe8dc161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.504 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e3f24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8dc09b100 0x7fe8dc158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.504 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e3f24640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc15ed30 0x7fe8dc0b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.504 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e3f24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc06d900 con 0x7fe8dc09b100 2024-04-17T06:08:57.504 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e3f24640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc0ccf50 con 0x7fe8dc15ed30 2024-04-17T06:08:57.504 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e3f24640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe8dc0cdc90 con 0x7fe8dc06d4b0 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e1cc0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8dc06d4b0 0x7fe8dc161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e24c1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc15ed30 0x7fe8dc0b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e1cc0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8dc06d4b0 0x7fe8dc161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:36748/0 (socket says 172.21.15.190:36748) 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e1cc0640 1 -- 172.21.15.190:0/16685035 learned_addr learned my addr 172.21.15.190:0/16685035 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e14bf640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8dc09b100 0x7fe8dc158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e24c1640 1 -- 172.21.15.190:0/16685035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8dc06d4b0 msgr2=0x7fe8dc161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e24c1640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8dc06d4b0 0x7fe8dc161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e24c1640 1 -- 172.21.15.190:0/16685035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8dc09b100 msgr2=0x7fe8dc158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e24c1640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8dc09b100 0x7fe8dc158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e24c1640 1 -- 172.21.15.190:0/16685035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc0cb480 con 0x7fe8dc15ed30 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e1cc0640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8dc06d4b0 0x7fe8dc161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.504+0000 7fe8e24c1640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc15ed30 0x7fe8dc0b7300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4032000 tx=0x7fe8d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.505 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.505+0000 7fe8ce7fc640 1 -- 172.21.15.190:0/16685035 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4043020 con 0x7fe8dc15ed30 2024-04-17T06:08:57.506 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.505+0000 7fe8ce7fc640 1 -- 172.21.15.190:0/16685035 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d403e450 con 0x7fe8dc15ed30 2024-04-17T06:08:57.506 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.505+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8dc159280 con 0x7fe8dc15ed30 2024-04-17T06:08:57.506 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.505+0000 7fe8ce7fc640 1 -- 172.21.15.190:0/16685035 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4009510 con 0x7fe8dc15ed30 2024-04-17T06:08:57.506 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.505+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8dc0cba20 con 0x7fe8dc15ed30 2024-04-17T06:08:57.506 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.505+0000 7fe8ce7fc640 1 -- 172.21.15.190:0/16685035 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8d4046020 con 0x7fe8dc15ed30 2024-04-17T06:08:57.506 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.505+0000 7fe8ce7fc640 1 -- 172.21.15.190:0/16685035 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe8d4048980 con 0x7fe8dc15ed30 2024-04-17T06:08:57.506 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.505+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8dc068a20 con 0x7fe8dc15ed30 2024-04-17T06:08:57.506 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.507+0000 7fe8ce7fc640 1 -- 172.21.15.190:0/16685035 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe8d404d030 con 0x7fe8dc15ed30 2024-04-17T06:08:57.578 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.580+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "79cd6222-7c39-4b17-bf01-253339bd1678", "id": 5} v 0) -- 0x7fe8dc06d140 con 0x7fe8dc15ed30 2024-04-17T06:08:57.583 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.585+0000 7fe8ce7fc640 1 -- 172.21.15.190:0/16685035 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "79cd6222-7c39-4b17-bf01-253339bd1678", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe8d402ca90 con 0x7fe8dc15ed30 2024-04-17T06:08:57.583 INFO:teuthology.orchestra.run.smithi190.stdout:5 2024-04-17T06:08:57.584 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc15ed30 msgr2=0x7fe8dc0b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.584 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc15ed30 0x7fe8dc0b7300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4032000 tx=0x7fe8d4004a40 comp rx=0 tx=0).stop 2024-04-17T06:08:57.584 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 shutdown_connections 2024-04-17T06:08:57.585 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe8dc15ed30 0x7fe8dc0b7300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.585 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8dc09b100 0x7fe8dc158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.585 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 --2- 172.21.15.190:0/16685035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8dc06d4b0 0x7fe8dc161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.585 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 >> 172.21.15.190:0/16685035 conn(0x7fe8dc0b41a0 msgr2=0x7fe8dc0af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.585 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 shutdown_connections 2024-04-17T06:08:57.585 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.586+0000 7fe8e3f24640 1 -- 172.21.15.190:0/16685035 wait complete. 2024-04-17T06:08:57.594 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 92a99e45-dcb9-43bc-98cc-650deff014ad 6 2024-04-17T06:08:57.739 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.741+0000 7f8b62931640 1 Processor -- start 2024-04-17T06:08:57.740 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.741+0000 7f8b62931640 1 -- start start 2024-04-17T06:08:57.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b62931640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0b19a0 0x7f8b5c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b62931640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b5c0d0ea0 0x7f8b5c09af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b62931640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8b5c06caa0 0x7f8b5c0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b62931640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8b5c0d1940 con 0x7f8b5c06caa0 2024-04-17T06:08:57.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b62931640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f8b5c0cd4a0 con 0x7f8b5c0d0ea0 2024-04-17T06:08:57.741 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b62931640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5c0b1d80 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b5b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b5c0d0ea0 0x7f8b5c09af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b5bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0b19a0 0x7f8b5c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b5b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b5c0d0ea0 0x7f8b5c09af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45128/0 (socket says 172.21.15.190:45128) 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b5bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0b19a0 0x7f8b5c09a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:36760/0 (socket says 172.21.15.190:36760) 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.742+0000 7f8b5b7fe640 1 -- 172.21.15.190:0/2260728319 learned_addr learned my addr 172.21.15.190:0/2260728319 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.743+0000 7f8b60ece640 1 --2- 172.21.15.190:0/2260728319 >> v2:172.21.15.135:3300/0 conn(0x7f8b5c06caa0 0x7f8b5c0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.743+0000 7f8b5bfff640 1 -- 172.21.15.190:0/2260728319 >> v2:172.21.15.190:3300/0 conn(0x7f8b5c0d0ea0 msgr2=0x7f8b5c09af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.743+0000 7f8b5bfff640 1 --2- 172.21.15.190:0/2260728319 >> v2:172.21.15.190:3300/0 conn(0x7f8b5c0d0ea0 0x7f8b5c09af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.743+0000 7f8b5bfff640 1 -- 172.21.15.190:0/2260728319 >> v2:172.21.15.135:3300/0 conn(0x7f8b5c06caa0 msgr2=0x7f8b5c0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.743+0000 7f8b5bfff640 1 --2- 172.21.15.190:0/2260728319 >> v2:172.21.15.135:3300/0 conn(0x7f8b5c06caa0 0x7f8b5c0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.742 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.743+0000 7f8b5bfff640 1 -- 172.21.15.190:0/2260728319 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c0d1380 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.744+0000 7f8b5bfff640 1 --2- 172.21.15.190:0/2260728319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0b19a0 0x7f8b5c09a9f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c073e10 tx=0x7f8b4c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd91ccffe810183c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.744+0000 7f8b5affd640 1 -- 172.21.15.190:0/2260728319 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0315a0 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.744+0000 7f8b5affd640 1 -- 172.21.15.190:0/2260728319 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4c037070 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.744+0000 7f8b5affd640 1 -- 172.21.15.190:0/2260728319 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c031960 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 -- 172.21.15.190:0/2260728319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0b19a0 msgr2=0x7f8b5c09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 --2- 172.21.15.190:0/2260728319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0b19a0 0x7f8b5c09a9f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c073e10 tx=0x7f8b4c02b840 comp rx=0 tx=0).stop 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 -- 172.21.15.190:0/2260728319 shutdown_connections 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 --2- 172.21.15.190:0/2260728319 >> v2:172.21.15.135:3300/0 conn(0x7f8b5c06caa0 0x7f8b5c0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.744 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 --2- 172.21.15.190:0/2260728319 >> v2:172.21.15.190:3300/0 conn(0x7f8b5c0d0ea0 0x7f8b5c09af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.745 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 --2- 172.21.15.190:0/2260728319 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0b19a0 0x7f8b5c09a9f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.745 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 -- 172.21.15.190:0/2260728319 >> 172.21.15.190:0/2260728319 conn(0x7f8b5c0b41a0 msgr2=0x7f8b5c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.745 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 -- 172.21.15.190:0/2260728319 shutdown_connections 2024-04-17T06:08:57.745 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.745+0000 7f8b62931640 1 -- 172.21.15.190:0/2260728319 wait complete. 2024-04-17T06:08:57.745 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.746+0000 7f8b62931640 1 Processor -- start 2024-04-17T06:08:57.745 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.746+0000 7f8b62931640 1 -- start start 2024-04-17T06:08:57.745 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.748+0000 7f6d2441a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b62931640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b5c06caa0 0x7f8b5c158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b62931640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b5c0b19a0 0x7f8b5c0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b62931640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0d0ea0 0x7f8b5c0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b62931640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c0cda20 con 0x7f8b5c06caa0 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b62931640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c0cd740 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b62931640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5c0ce480 con 0x7f8b5c0d0ea0 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b5c0b19a0 0x7f8b5c0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b60ece640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0d0ea0 0x7f8b5c0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.748 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b5c0b19a0 0x7f8b5c0c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45134/0 (socket says 172.21.15.190:45134) 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 -- 172.21.15.190:0/681563285 learned_addr learned my addr 172.21.15.190:0/681563285 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5bfff640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b5c06caa0 0x7f8b5c158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 -- 172.21.15.190:0/681563285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0d0ea0 msgr2=0x7f8b5c0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0d0ea0 0x7f8b5c0c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 -- 172.21.15.190:0/681563285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b5c06caa0 msgr2=0x7f8b5c158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b5c06caa0 0x7f8b5c158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 -- 172.21.15.190:0/681563285 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c0b7150 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b60ece640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0d0ea0 0x7f8b5c0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.747+0000 7f8b5b7fe640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b5c0b19a0 0x7f8b5c0c79e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400e870 tx=0x7f8b4400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.748+0000 7f8b58ff9640 1 -- 172.21.15.190:0/681563285 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44017070 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.748+0000 7f8b58ff9640 1 -- 172.21.15.190:0/681563285 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b4400c8e0 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.748+0000 7f8b58ff9640 1 -- 172.21.15.190:0/681563285 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400ccd0 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.748+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5c0b7430 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.748+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5c0b79f0 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.748+0000 7f8b58ff9640 1 -- 172.21.15.190:0/681563285 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b4401a070 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.748+0000 7f8b58ff9640 1 -- 172.21.15.190:0/681563285 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f8b4401cad0 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.749+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b5c068a20 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.750 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.751+0000 7f8b58ff9640 1 -- 172.21.15.190:0/681563285 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8b4401c4e0 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.814 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.816+0000 7f67df814040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-17T06:08:57.833 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.835+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "92a99e45-dcb9-43bc-98cc-650deff014ad", "id": 6} v 0) -- 0x7f8b5c0b8570 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.842 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.844+0000 7f8b58ff9640 1 -- 172.21.15.190:0/681563285 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92a99e45-dcb9-43bc-98cc-650deff014ad", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f8b4401c7b0 con 0x7f8b5c0b19a0 2024-04-17T06:08:57.842 INFO:teuthology.orchestra.run.smithi190.stdout:6 2024-04-17T06:08:57.843 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b5c0b19a0 msgr2=0x7f8b5c0c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.843 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b5c0b19a0 0x7f8b5c0c79e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400e870 tx=0x7f8b4400ed30 comp rx=0 tx=0).stop 2024-04-17T06:08:57.843 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 shutdown_connections 2024-04-17T06:08:57.843 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b5c0d0ea0 0x7f8b5c0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.843 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b5c0b19a0 0x7f8b5c0c79e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.843 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 --2- 172.21.15.190:0/681563285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b5c06caa0 0x7f8b5c158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 >> 172.21.15.190:0/681563285 conn(0x7f8b5c0b41a0 msgr2=0x7f8b5c0b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 shutdown_connections 2024-04-17T06:08:57.844 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.845+0000 7f8b62931640 1 -- 172.21.15.190:0/681563285 wait complete. 2024-04-17T06:08:57.853 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3 7 2024-04-17T06:08:57.880 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.882+0000 7f6d2441a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T06:08:57.957 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.959+0000 7f5b4547e640 1 Processor -- start 2024-04-17T06:08:57.957 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.959+0000 7f5b4547e640 1 -- start start 2024-04-17T06:08:57.957 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:57.959+0000 7f67df814040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.959+0000 7f5b4547e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b400cfed0 0x7f5b40073a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.959+0000 7f5b4547e640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b40073f50 con 0x7f5b40070880 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.959+0000 7f5b4547e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b400d0420 con 0x7f5b4015b9f0 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.959+0000 7f5b4547e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f5b400d0700 con 0x7f5b400cfed0 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.960+0000 7f5b37fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b400cfed0 0x7f5b40073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.960+0000 7f5b37fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5b400cfed0 0x7f5b40073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45146/0 (socket says 172.21.15.190:45146) 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.960+0000 7f5b37fff640 1 -- 172.21.15.190:0/4276293762 learned_addr learned my addr 172.21.15.190:0/4276293762 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.960+0000 7f5b37fff640 1 -- 172.21.15.190:0/4276293762 >> v1:172.21.15.190:6789/0 conn(0x7f5b4015b9f0 legacy=0x7f5b400b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.960+0000 7f5b37fff640 1 -- 172.21.15.190:0/4276293762 >> v1:172.21.15.135:6789/0 conn(0x7f5b40070880 legacy=0x7f5b400ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.960+0000 7f5b37fff640 1 -- 172.21.15.190:0/4276293762 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b400cf780 con 0x7f5b400cfed0 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.961+0000 7f5b37fff640 1 --2- 172.21.15.190:0/4276293762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b400cfed0 0x7f5b40073a10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5b280098a0 tx=0x7f5b2802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9d177adc6f7adb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.961+0000 7f5b3e7fc640 1 -- 172.21.15.190:0/4276293762 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28034070 con 0x7f5b400cfed0 2024-04-17T06:08:57.960 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.961+0000 7f5b3e7fc640 1 -- 172.21.15.190:0/4276293762 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b28030840 con 0x7f5b400cfed0 2024-04-17T06:08:57.960 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.961+0000 7f5b3e7fc640 1 -- 172.21.15.190:0/4276293762 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28030c10 con 0x7f5b400cfed0 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.962+0000 7f5b4547e640 1 -- 172.21.15.190:0/4276293762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b400cfed0 msgr2=0x7f5b40073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.962+0000 7f5b4547e640 1 --2- 172.21.15.190:0/4276293762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b400cfed0 0x7f5b40073a10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5b280098a0 tx=0x7f5b2802b980 comp rx=0 tx=0).stop 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.962+0000 7f5b4547e640 1 -- 172.21.15.190:0/4276293762 shutdown_connections 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.962+0000 7f5b4547e640 1 --2- 172.21.15.190:0/4276293762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b400cfed0 0x7f5b40073a10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.962+0000 7f5b4547e640 1 -- 172.21.15.190:0/4276293762 >> 172.21.15.190:0/4276293762 conn(0x7f5b400b3320 msgr2=0x7f5b400b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.962+0000 7f5b4547e640 1 -- 172.21.15.190:0/4276293762 shutdown_connections 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.962+0000 7f5b4547e640 1 -- 172.21.15.190:0/4276293762 wait complete. 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.963+0000 7f5b4547e640 1 Processor -- start 2024-04-17T06:08:57.961 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.963+0000 7f5b4547e640 1 -- start start 2024-04-17T06:08:57.964 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.963+0000 7f5b4547e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b40070880 0x7f5b40149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.964 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.963+0000 7f5b4547e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b400cfed0 0x7f5b4014b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.964 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b4547e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4015b9f0 0x7f5b401f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:57.964 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b4547e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40070d30 con 0x7f5b400cfed0 2024-04-17T06:08:57.964 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b4547e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5b400cf530 con 0x7f5b40070880 2024-04-17T06:08:57.964 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b4547e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5b40070eb0 con 0x7f5b4015b9f0 2024-04-17T06:08:57.964 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b40070880 0x7f5b40149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.964 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b40070880 0x7f5b40149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:45160/0 (socket says 172.21.15.190:45160) 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 -- 172.21.15.190:0/3895963651 learned_addr learned my addr 172.21.15.190:0/3895963651 (peer_addr_for_me v2:172.21.15.190:0/0) 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3f7fe640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4015b9f0 0x7f5b401f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b37fff640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b400cfed0 0x7f5b4014b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 -- 172.21.15.190:0/3895963651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4015b9f0 msgr2=0x7f5b401f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4015b9f0 0x7f5b401f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 -- 172.21.15.190:0/3895963651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b400cfed0 msgr2=0x7f5b4014b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b400cfed0 0x7f5b4014b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 -- 172.21.15.190:0/3895963651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4014cbd0 con 0x7f5b40070880 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b3effd640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b40070880 0x7f5b40149db0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c400 tx=0x7f5b3000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b377fe640 1 -- 172.21.15.190:0/3895963651 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30017070 con 0x7f5b40070880 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b377fe640 1 -- 172.21.15.190:0/3895963651 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b300052d0 con 0x7f5b40070880 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b377fe640 1 -- 172.21.15.190:0/3895963651 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30003830 con 0x7f5b40070880 2024-04-17T06:08:57.965 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4014e8c0 con 0x7f5b40070880 2024-04-17T06:08:57.966 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4014ee80 con 0x7f5b40070880 2024-04-17T06:08:57.966 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.964+0000 7f5b377fe640 1 -- 172.21.15.190:0/3895963651 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b3001a070 con 0x7f5b40070880 2024-04-17T06:08:57.966 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.965+0000 7f5b377fe640 1 -- 172.21.15.190:0/3895963651 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f5b30011b80 con 0x7f5b40070880 2024-04-17T06:08:57.966 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.965+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b40068a20 con 0x7f5b40070880 2024-04-17T06:08:57.966 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:57.967+0000 7f5b377fe640 1 -- 172.21.15.190:0/3895963651 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5b30021020 con 0x7f5b40070880 2024-04-17T06:08:57.991 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:57.993+0000 7f6d2441a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T06:08:58.036 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:58.038+0000 7f6d2441a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T06:08:58.038 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.040+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3", "id": 7} v 0) -- 0x7f5b400cf210 con 0x7f5b40070880 2024-04-17T06:08:58.047 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.049+0000 7f5b377fe640 1 -- 172.21.15.190:0/3895963651 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f5b30011e20 con 0x7f5b40070880 2024-04-17T06:08:58.047 INFO:teuthology.orchestra.run.smithi190.stdout:7 2024-04-17T06:08:58.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b40070880 msgr2=0x7f5b40149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:58.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b40070880 0x7f5b40149db0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c400 tx=0x7f5b3000c8c0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 shutdown_connections 2024-04-17T06:08:58.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5b4015b9f0 0x7f5b401f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b400cfed0 0x7f5b4014b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 --2- 172.21.15.190:0/3895963651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5b40070880 0x7f5b40149db0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.048 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 >> 172.21.15.190:0/3895963651 conn(0x7f5b400b3320 msgr2=0x7f5b400b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:58.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 shutdown_connections 2024-04-17T06:08:58.049 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:08:58.050+0000 7f5b4547e640 1 -- 172.21.15.190:0/3895963651 wait complete. 2024-04-17T06:08:58.058 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-17T06:08:58.059 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-17T06:08:58.061 INFO:tasks.ceph.osd.0:Started 2024-04-17T06:08:58.061 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-17T06:08:58.061 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-17T06:08:58.066 INFO:tasks.ceph.osd.1:Started 2024-04-17T06:08:58.066 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-17T06:08:58.066 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-17T06:08:58.068 INFO:tasks.ceph.osd.2:Started 2024-04-17T06:08:58.068 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-17T06:08:58.069 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-17T06:08:58.071 INFO:tasks.ceph.osd.3:Started 2024-04-17T06:08:58.071 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-17T06:08:58.071 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-17T06:08:58.073 INFO:tasks.ceph.osd.4:Started 2024-04-17T06:08:58.073 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-17T06:08:58.073 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-17T06:08:58.075 INFO:tasks.ceph.osd.5:Started 2024-04-17T06:08:58.075 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-17T06:08:58.075 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-17T06:08:58.078 INFO:tasks.ceph.osd.6:Started 2024-04-17T06:08:58.078 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-17T06:08:58.078 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-17T06:08:58.079 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:58.081+0000 7f67df814040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-17T06:08:58.080 INFO:tasks.ceph.osd.7:Started 2024-04-17T06:08:58.081 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:08:58.135 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:58.137+0000 7f67df814040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-17T06:08:58.187 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:58.189+0000 7f6d2441a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:08:58.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.214+0000 7fd56578f640 1 Processor -- start 2024-04-17T06:08:58.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.215+0000 7fd56578f640 1 -- start start 2024-04-17T06:08:58.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.215+0000 7fd56578f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5600d07e0 0x7fd56015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:58.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.215+0000 7fd56578f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d0bc0 0x7fd5600d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:58.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.215+0000 7fd56578f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd56009ba00 con 0x7fd5601604d0 2024-04-17T06:08:58.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.215+0000 7fd56578f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd5600cd270 con 0x7fd5600d07e0 2024-04-17T06:08:58.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.215+0000 7fd56578f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd5600cd4a0 con 0x7fd5600d0bc0 2024-04-17T06:08:58.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55f7fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd5601604d0 0x7fd560160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57322/0 (socket says 172.21.15.135:57322) 2024-04-17T06:08:58.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55f7fe640 1 -- 172.21.15.135:0/4151975885 learned_addr learned my addr 172.21.15.135:0/4151975885 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:08:58.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55effd640 1 --2- 172.21.15.135:0/4151975885 >> v2:172.21.15.190:3300/0 conn(0x7fd5600d07e0 0x7fd56015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd557fff640 1 --2- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d0bc0 0x7fd5600d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55e7fc640 1 -- 172.21.15.135:0/4151975885 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200279512 0 0) 0x7fd56009ba00 con 0x7fd5601604d0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55e7fc640 1 -- 172.21.15.135:0/4151975885 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd54c003520 con 0x7fd5601604d0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55effd640 1 -- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d0bc0 msgr2=0x7fd5600d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55effd640 1 --2- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d0bc0 0x7fd5600d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55effd640 1 -- 172.21.15.135:0/4151975885 >> v1:172.21.15.135:6789/0 conn(0x7fd5601604d0 legacy=0x7fd560160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd55effd640 1 -- 172.21.15.135:0/4151975885 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5600cd780 con 0x7fd5600d07e0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.216+0000 7fd557fff640 1 --2- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d0bc0 0x7fd5600d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.217+0000 7fd55effd640 1 --2- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 0x7fd56015d9e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd5500097a0 tx=0x7fd55002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81d13af797aa02bf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.217+0000 7fd55e7fc640 1 -- 172.21.15.135:0/4151975885 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500315a0 con 0x7fd5600d07e0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.217+0000 7fd55e7fc640 1 -- 172.21.15.135:0/4151975885 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd550037070 con 0x7fd5600d07e0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.217+0000 7fd55e7fc640 1 -- 172.21.15.135:0/4151975885 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550031940 con 0x7fd5600d07e0 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.218+0000 7fd56578f640 1 -- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 msgr2=0x7fd56015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:58.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.218+0000 7fd56578f640 1 --2- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 0x7fd56015d9e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd5500097a0 tx=0x7fd55002b840 comp rx=0 tx=0).stop 2024-04-17T06:08:58.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.218+0000 7fd56578f640 1 -- 172.21.15.135:0/4151975885 shutdown_connections 2024-04-17T06:08:58.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.218+0000 7fd56578f640 1 --2- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5600d0bc0 0x7fd5600d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.218+0000 7fd56578f640 1 --2- 172.21.15.135:0/4151975885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 0x7fd56015d9e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.218+0000 7fd56578f640 1 -- 172.21.15.135:0/4151975885 >> 172.21.15.135:0/4151975885 conn(0x7fd5600b4250 msgr2=0x7fd5600b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:58.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.218+0000 7fd56578f640 1 -- 172.21.15.135:0/4151975885 shutdown_connections 2024-04-17T06:08:58.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.218+0000 7fd56578f640 1 -- 172.21.15.135:0/4151975885 wait complete. 2024-04-17T06:08:58.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.219+0000 7fd56578f640 1 Processor -- start 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.219+0000 7fd56578f640 1 -- start start 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd56578f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 0x7fd5600cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd56578f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5600d0bc0 0x7fd560157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd56578f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5601604d0 0x7fd560158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd56578f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5600ce020 con 0x7fd5600d0bc0 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd56578f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd5600cdd40 con 0x7fd5600d07e0 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd56578f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd5600d1560 con 0x7fd5601604d0 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd557fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5600d0bc0 0x7fd560157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd55effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 0x7fd5600cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd557fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5600d0bc0 0x7fd560157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54500/0 (socket says 172.21.15.135:54500) 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd557fff640 1 -- 172.21.15.135:0/567299390 learned_addr learned my addr 172.21.15.135:0/567299390 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd55f7fe640 1 --2- 172.21.15.135:0/567299390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5601604d0 0x7fd560158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd55effd640 1 -- 172.21.15.135:0/567299390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5601604d0 msgr2=0x7fd560158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:58.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd55effd640 1 --2- 172.21.15.135:0/567299390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5601604d0 0x7fd560158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd55effd640 1 -- 172.21.15.135:0/567299390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5600d0bc0 msgr2=0x7fd560157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd55effd640 1 --2- 172.21.15.135:0/567299390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5600d0bc0 0x7fd560157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd55effd640 1 -- 172.21.15.135:0/567299390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5600b61a0 con 0x7fd5600d07e0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd55effd640 1 --2- 172.21.15.135:0/567299390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 0x7fd5600cbd00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd550033000 tx=0x7fd5500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd5577fe640 1 -- 172.21.15.135:0/567299390 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550032430 con 0x7fd5600d07e0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd5577fe640 1 -- 172.21.15.135:0/567299390 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5500325d0 con 0x7fd5600d07e0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd5577fe640 1 -- 172.21.15.135:0/567299390 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500329a0 con 0x7fd5600d07e0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.220+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5600b6480 con 0x7fd5600d07e0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.221+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5600b6a40 con 0x7fd5600d07e0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.221+0000 7fd5577fe640 1 -- 172.21.15.135:0/567299390 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd55004e430 con 0x7fd5600d07e0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.221+0000 7fd5577fe640 1 -- 172.21.15.135:0/567299390 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd55002f030 con 0x7fd5600d07e0 2024-04-17T06:08:58.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.221+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd560068a20 con 0x7fd5600d07e0 2024-04-17T06:08:58.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.224+0000 7fd5577fe640 1 -- 172.21.15.135:0/567299390 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd550040070 con 0x7fd5600d07e0 2024-04-17T06:08:58.255 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:58.258+0000 7f67df814040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:08:58.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.284+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5601587a0 con 0x7fd5600d07e0 2024-04-17T06:08:58.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.285+0000 7fd5577fe640 1 -- 172.21.15.135:0/567299390 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd55004e720 con 0x7fd5600d07e0 2024-04-17T06:08:58.283 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:08:58.283 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":10,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:08:58.042387+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.285+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 msgr2=0x7fd5600cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.285+0000 7fd56578f640 1 --2- 172.21.15.135:0/567299390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 0x7fd5600cbd00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd550033000 tx=0x7fd5500046c0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.286+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 shutdown_connections 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.286+0000 7fd56578f640 1 --2- 172.21.15.135:0/567299390 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5601604d0 0x7fd560158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.286+0000 7fd56578f640 1 --2- 172.21.15.135:0/567299390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5600d0bc0 0x7fd560157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.286+0000 7fd56578f640 1 --2- 172.21.15.135:0/567299390 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5600d07e0 0x7fd5600cbd00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.286+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 >> 172.21.15.135:0/567299390 conn(0x7fd5600b4250 msgr2=0x7fd560160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.286+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 shutdown_connections 2024-04-17T06:08:58.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:08:58.286+0000 7fd56578f640 1 -- 172.21.15.135:0/567299390 wait complete. 2024-04-17T06:08:58.286 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:58.289+0000 7f6d2441a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T06:08:58.295 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-17T06:08:58.295 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-17T06:08:58.350 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:58.352+0000 7f67df814040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-17T06:08:58.445 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:58.447+0000 7f6d2441a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:08:58.488 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:58.490+0000 7f6d2441a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T06:08:58.505 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:58.507+0000 7f67df814040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-17T06:08:58.546 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:58.548+0000 7f67df814040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-17T06:08:58.642 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:58.644+0000 7f6d2441a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T06:08:58.669 INFO:tasks.ceph.osd.1.smithi135.stderr:2024-04-17T06:08:58.671+0000 7f3799415640 -1 Falling back to public interface 2024-04-17T06:08:58.670 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-04-17T06:08:58.672+0000 7ffa1c29b640 -1 Falling back to public interface 2024-04-17T06:08:58.678 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-04-17T06:08:58.680+0000 7faf63ea4640 -1 Falling back to public interface 2024-04-17T06:08:58.698 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-04-17T06:08:58.700+0000 7f7d7ae13640 -1 Falling back to public interface 2024-04-17T06:08:58.702 INFO:tasks.ceph.osd.0.smithi135.stderr:2024-04-17T06:08:58.701+0000 7fcedb8bf640 -1 Falling back to public interface 2024-04-17T06:08:58.702 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-04-17T06:08:58.701+0000 7fde64a0b640 -1 Falling back to public interface 2024-04-17T06:08:58.703 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:58.703+0000 7f67df814040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-17T06:08:58.703 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-17T06:08:58.702+0000 7fca4521b640 -1 Falling back to public interface 2024-04-17T06:08:58.703 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-17T06:08:58.703+0000 7fca43c05640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6816/0: (98) Address already in use 2024-04-17T06:08:58.703 INFO:tasks.ceph.osd.3.smithi135.stderr:2024-04-17T06:08:58.704+0000 7fdbaf171640 -1 Falling back to public interface 2024-04-17T06:08:58.704 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-17T06:08:58.705+0000 7fca43c05640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6822/0: (98) Address already in use 2024-04-17T06:08:58.704 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-17T06:08:58.706+0000 7fca43c05640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6827/0: (98) Address already in use 2024-04-17T06:08:59.101 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T06:08:59.193 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.194+0000 7f7a9f093640 1 Processor -- start 2024-04-17T06:08:59.193 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.194+0000 7f7a9f093640 1 -- start start 2024-04-17T06:08:59.197 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.195+0000 7f7a9f093640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 0x7f7a980b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.195+0000 7f7a9f093640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a9815cdd0 con 0x7f7a980cd4b0 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.195+0000 7f7a9f093640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a98070380 con 0x7f7a980b1e40 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.195+0000 7f7a9f093640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7a98070660 con 0x7f7a9815c9f0 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.195+0000 7f7a9d630640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7a980cd4b0 0x7f7a980d0ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:57334/0 (socket says 172.21.15.135:57334) 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.195+0000 7f7a9d630640 1 -- 172.21.15.135:0/3025243783 learned_addr learned my addr 172.21.15.135:0/3025243783 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9ce2f640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 0x7f7a980b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551167539 0 0) 0x7f7a9815cdd0 con 0x7f7a980cd4b0 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a80003520 con 0x7f7a980cd4b0 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4239277162 0 0) 0x7f7a80003520 con 0x7f7a980cd4b0 2024-04-17T06:08:59.198 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 msgr2=0x7f7a980b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 0x7f7a980b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 >> v1:172.21.15.190:6789/0 conn(0x7f7a980b1e40 legacy=0x7f7a980ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a98070940 con 0x7f7a980cd4b0 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9ce2f640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 0x7f7a980b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f7a900019f0 con 0x7f7a980cd4b0 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 >> v1:172.21.15.135:6789/0 conn(0x7f7a980cd4b0 legacy=0x7f7a980d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a800039e0 0x7f7a80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 0x7f7a80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a80004d40 0x7f7a800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98070660 con 0x7f7a800039e0 2024-04-17T06:08:59.199 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8000ba10 con 0x7f7a80004d40 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8000bc10 con 0x7f7a9815c9f0 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9ce2f640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a800039e0 0x7f7a80003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a8ffff640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 0x7f7a80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9d630640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a80004d40 0x7f7a800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9ce2f640 1 -- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 msgr2=0x7f7a80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9ce2f640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a9815c9f0 0x7f7a80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9ce2f640 1 -- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a80004d40 msgr2=0x7f7a800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9ce2f640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a80004d40 0x7f7a800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9ce2f640 1 -- 172.21.15.135:0/3025243783 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a80003520 con 0x7f7a800039e0 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.196+0000 7f7a9d630640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a80004d40 0x7f7a800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:08:59.200 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.197+0000 7f7a9ce2f640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a800039e0 0x7f7a80003dc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009c90 tx=0x7f7a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9391662a0f79abe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.197+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94039070 con 0x7f7a800039e0 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.197+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9402c8f0 con 0x7f7a800039e0 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.197+0000 7f7a8f7fe640 1 -- 172.21.15.135:0/3025243783 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94009860 con 0x7f7a800039e0 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.198+0000 7f7a9f093640 1 -- 172.21.15.135:0/3025243783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a800039e0 msgr2=0x7f7a80003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.198+0000 7f7a9f093640 1 --2- 172.21.15.135:0/3025243783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a800039e0 0x7f7a80003dc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009c90 tx=0x7f7a80005830 comp rx=0 tx=0).stop 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.198+0000 7f7a9d630640 1 -- 172.21.15.135:0/3025243783 reap_dead start 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.198+0000 7f7a9f093640 1 -- 172.21.15.135:0/3025243783 shutdown_connections 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.198+0000 7f7a9f093640 1 -- 172.21.15.135:0/3025243783 >> 172.21.15.135:0/3025243783 conn(0x7f7a98071c20 msgr2=0x7f7a98072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.198+0000 7f7a9f093640 1 -- 172.21.15.135:0/3025243783 shutdown_connections 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.198+0000 7f7a9f093640 1 -- 172.21.15.135:0/3025243783 wait complete. 2024-04-17T06:08:59.201 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.199+0000 7f7a9f093640 1 Processor -- start 2024-04-17T06:08:59.202 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.199+0000 7f7a9f093640 1 -- start start 2024-04-17T06:08:59.204 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a9f093640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a800039e0 0x7f7a981f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:59.204 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a9f093640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a9814b9c0 0x7f7a980cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:59.204 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a9f093640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a980b2250 0x7f7a980b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:08:59.204 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a9f093640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9806ffa0 con 0x7f7a9814b9c0 2024-04-17T06:08:59.204 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a9f093640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98070f00 con 0x7f7a980b2250 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a9f093640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7a98070120 con 0x7f7a800039e0 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a9814b9c0 0x7f7a980cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a9814b9c0 0x7f7a980cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54512/0 (socket says 172.21.15.135:54512) 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a8ffff640 1 -- 172.21.15.135:0/3900223223 learned_addr learned my addr 172.21.15.135:0/3900223223 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a9ce2f640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a800039e0 0x7f7a981f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.200+0000 7f7a9d630640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a980b2250 0x7f7a980b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8ffff640 1 -- 172.21.15.135:0/3900223223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a800039e0 msgr2=0x7f7a981f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8ffff640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a800039e0 0x7f7a981f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8ffff640 1 -- 172.21.15.135:0/3900223223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a980b2250 msgr2=0x7f7a980b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8ffff640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a980b2250 0x7f7a980b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.205 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8ffff640 1 -- 172.21.15.135:0/3900223223 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9815ef60 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8ffff640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a9814b9c0 0x7f7a980cc260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800f7b0 tx=0x7f7a8800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8d7fa640 1 -- 172.21.15.135:0/3900223223 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880050d0 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8d7fa640 1 -- 172.21.15.135:0/3900223223 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a88018070 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9815f240 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8d7fa640 1 -- 172.21.15.135:0/3900223223 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8800d730 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a980cc7a0 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8d7fa640 1 -- 172.21.15.135:0/3900223223 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a8800d8d0 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.201+0000 7f7a8d7fa640 1 -- 172.21.15.135:0/3900223223 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7a88003d70 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.202+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a9804fd50 con 0x7f7a9814b9c0 2024-04-17T06:08:59.206 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.205+0000 7f7a8d7fa640 1 -- 172.21.15.135:0/3900223223 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7a8800dd50 con 0x7f7a9814b9c0 2024-04-17T06:08:59.266 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.268+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a980b5510 con 0x7f7a9814b9c0 2024-04-17T06:08:59.266 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.268+0000 7f7a8d7fa640 1 -- 172.21.15.135:0/3900223223 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7a880035f0 con 0x7f7a9814b9c0 2024-04-17T06:08:59.266 INFO:teuthology.misc.health.smithi135.stdout: 2024-04-17T06:08:59.267 INFO:teuthology.misc.health.smithi135.stdout:{"epoch":10,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:08:58.042387+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:08:59.267 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.269+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a9814b9c0 msgr2=0x7f7a980cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:08:59.268 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.269+0000 7f7a9f093640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a9814b9c0 0x7f7a980cc260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800f7b0 tx=0x7f7a8800fbb0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.268 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.269+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 shutdown_connections 2024-04-17T06:08:59.268 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.269+0000 7f7a9f093640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a980b2250 0x7f7a980b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.268 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.269+0000 7f7a9f093640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a9814b9c0 0x7f7a980cc260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.268 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.269+0000 7f7a9f093640 1 --2- 172.21.15.135:0/3900223223 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a800039e0 0x7f7a981f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:08:59.268 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.269+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 >> 172.21.15.135:0/3900223223 conn(0x7f7a98071c20 msgr2=0x7f7a980d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:08:59.268 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.269+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 shutdown_connections 2024-04-17T06:08:59.268 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:08:59.270+0000 7f7a9f093640 1 -- 172.21.15.135:0/3900223223 wait complete. 2024-04-17T06:08:59.277 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-17T06:08:59.484 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:59.486+0000 7f6d2441a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T06:08:59.517 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:59.519+0000 7f67df814040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-17T06:08:59.759 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:59.762+0000 7f6d2441a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T06:08:59.793 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:59.795+0000 7f67df814040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-17T06:08:59.811 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:59.813+0000 7f6d2441a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T06:08:59.843 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:59.846+0000 7f67df814040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-17T06:08:59.925 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:08:59.927+0000 7f6d2441a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T06:08:59.961 INFO:tasks.ceph.mgr.x.smithi190.stderr:2024-04-17T06:08:59.963+0000 7f67df814040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-17T06:09:00.938 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:09:00.940+0000 7f6cc4c17640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-17T06:09:00.939 INFO:tasks.ceph.mgr.y.smithi135.stderr:2024-04-17T06:09:00.941+0000 7f6cc4c17640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-17T06:09:02.129 INFO:tasks.ceph.osd.1.smithi135.stderr:2024-04-17T06:09:02.131+0000 7f3799415640 -1 osd.1 0 log_to_monitors true 2024-04-17T06:09:02.141 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-04-17T06:09:02.144+0000 7ffa1c29b640 -1 osd.4 0 log_to_monitors true 2024-04-17T06:09:02.181 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-04-17T06:09:02.183+0000 7faf63ea4640 -1 osd.7 0 log_to_monitors true 2024-04-17T06:09:02.199 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-04-17T06:09:02.202+0000 7f7d7ae13640 -1 osd.5 0 log_to_monitors true 2024-04-17T06:09:02.202 INFO:tasks.ceph.osd.0.smithi135.stderr:2024-04-17T06:09:02.205+0000 7fcedb8bf640 -1 osd.0 0 log_to_monitors true 2024-04-17T06:09:02.214 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-04-17T06:09:02.217+0000 7fde64a0b640 -1 osd.6 0 log_to_monitors true 2024-04-17T06:09:02.276 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-17T06:09:02.278+0000 7fca4521b640 -1 osd.2 0 log_to_monitors true 2024-04-17T06:09:02.278 INFO:tasks.ceph.osd.3.smithi135.stderr:2024-04-17T06:09:02.281+0000 7fdbaf171640 -1 osd.3 0 log_to_monitors true 2024-04-17T06:09:05.905 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-04-17T06:09:05.906+0000 7fde609a1640 -1 osd.6 0 waiting for initial osdmap 2024-04-17T06:09:05.952 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-17T06:09:05.953+0000 7fca40bec640 -1 osd.2 0 waiting for initial osdmap 2024-04-17T06:09:05.965 INFO:tasks.ceph.osd.2.smithi135.stderr:2024-04-17T06:09:05.967+0000 7fca38a0c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T06:09:05.995 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-04-17T06:09:05.997+0000 7f7d76804640 -1 osd.5 0 waiting for initial osdmap 2024-04-17T06:09:05.997 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-04-17T06:09:05.998+0000 7ffa18231640 -1 osd.4 0 waiting for initial osdmap 2024-04-17T06:09:06.021 INFO:tasks.ceph.osd.1.smithi135.stderr:2024-04-17T06:09:06.023+0000 7f3794ddb640 -1 osd.1 0 waiting for initial osdmap 2024-04-17T06:09:06.035 INFO:tasks.ceph.osd.1.smithi135.stderr:2024-04-17T06:09:06.038+0000 7f378cbfb640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T06:09:06.051 INFO:tasks.ceph.osd.6.smithi190.stderr:2024-04-17T06:09:06.053+0000 7fde587c1640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T06:09:06.057 INFO:tasks.ceph.osd.5.smithi190.stderr:2024-04-17T06:09:06.059+0000 7f7d6e624640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T06:09:06.058 INFO:tasks.ceph.osd.4.smithi190.stderr:2024-04-17T06:09:06.059+0000 7ffa10051640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T06:09:06.060 INFO:tasks.ceph.osd.0.smithi135.stderr:2024-04-17T06:09:06.062+0000 7fced7855640 -1 osd.0 0 waiting for initial osdmap 2024-04-17T06:09:06.061 INFO:tasks.ceph.osd.3.smithi135.stderr:2024-04-17T06:09:06.064+0000 7fdbab91b640 -1 osd.3 0 waiting for initial osdmap 2024-04-17T06:09:06.072 INFO:tasks.ceph.osd.0.smithi135.stderr:2024-04-17T06:09:06.075+0000 7fcecf675640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T06:09:06.073 INFO:tasks.ceph.osd.3.smithi135.stderr:2024-04-17T06:09:06.075+0000 7fdba2f27640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T06:09:06.084 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:06.174 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.176+0000 7fc1da511640 1 Processor -- start 2024-04-17T06:09:06.175 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.177+0000 7fc1da511640 1 -- start start 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.177+0000 7fc1da511640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc1d415e890 0x7fc1d40b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.177+0000 7fc1da511640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d406c310 con 0x7fc1d415ec70 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.177+0000 7fc1da511640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d40b68a0 con 0x7fc1d4071020 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.177+0000 7fc1da511640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d406c580 con 0x7fc1d415e890 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.178+0000 7fc1cbfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc1d415ec70 0x7fc1d4070910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:59782/0 (socket says 172.21.15.135:59782) 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.178+0000 7fc1cbfff640 1 -- 172.21.15.135:0/3813719896 learned_addr learned my addr 172.21.15.135:0/3813719896 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.178+0000 7fc1d3fff640 1 --2- 172.21.15.135:0/3813719896 >> v2:172.21.15.190:3300/0 conn(0x7fc1d415e890 0x7fc1d40b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.178+0000 7fc1d37fe640 1 -- 172.21.15.135:0/3813719896 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130638865 0 0) 0x7fc1d406c310 con 0x7fc1d415ec70 2024-04-17T06:09:06.176 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.178+0000 7fc1d37fe640 1 -- 172.21.15.135:0/3813719896 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1c0003520 con 0x7fc1d415ec70 2024-04-17T06:09:06.177 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.178+0000 7fc1d3fff640 1 -- 172.21.15.135:0/3813719896 >> v1:172.21.15.190:6789/0 conn(0x7fc1d4071020 legacy=0x7fc1d41686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:06.177 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.178+0000 7fc1d3fff640 1 -- 172.21.15.135:0/3813719896 >> v1:172.21.15.135:6789/0 conn(0x7fc1d415ec70 legacy=0x7fc1d4070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:06.177 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.178+0000 7fc1d3fff640 1 -- 172.21.15.135:0/3813719896 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d406c860 con 0x7fc1d415e890 2024-04-17T06:09:06.178 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1d3fff640 1 --2- 172.21.15.135:0/3813719896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 0x7fc1d40b61e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40097a0 tx=0x7fc1c402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45121fde3d8d1d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:06.178 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1d37fe640 1 -- 172.21.15.135:0/3813719896 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4035020 con 0x7fc1d415e890 2024-04-17T06:09:06.178 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1d37fe640 1 -- 172.21.15.135:0/3813719896 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4031830 con 0x7fc1d415e890 2024-04-17T06:09:06.178 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1d37fe640 1 -- 172.21.15.135:0/3813719896 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4031be0 con 0x7fc1d415e890 2024-04-17T06:09:06.178 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1da511640 1 -- 172.21.15.135:0/3813719896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 msgr2=0x7fc1d40b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:06.179 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1da511640 1 --2- 172.21.15.135:0/3813719896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 0x7fc1d40b61e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40097a0 tx=0x7fc1c402b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.179 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1da511640 1 -- 172.21.15.135:0/3813719896 shutdown_connections 2024-04-17T06:09:06.179 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1da511640 1 --2- 172.21.15.135:0/3813719896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 0x7fc1d40b61e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.179 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1da511640 1 -- 172.21.15.135:0/3813719896 >> 172.21.15.135:0/3813719896 conn(0x7fc1d40730d0 msgr2=0x7fc1d40734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:06.179 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1da511640 1 -- 172.21.15.135:0/3813719896 shutdown_connections 2024-04-17T06:09:06.179 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.179+0000 7fc1da511640 1 -- 172.21.15.135:0/3813719896 wait complete. 2024-04-17T06:09:06.179 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1da511640 1 Processor -- start 2024-04-17T06:09:06.179 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1da511640 1 -- start start 2024-04-17T06:09:06.182 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1da511640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1d4071020 0x7fc1d4148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:06.182 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1da511640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 0x7fc1d41f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:06.182 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1da511640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d415ec70 0x7fc1d40cff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:06.182 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1da511640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d406df10 con 0x7fc1d4071020 2024-04-17T06:09:06.182 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1da511640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d406dd90 con 0x7fc1d415e890 2024-04-17T06:09:06.182 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1da511640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d40b53e0 con 0x7fc1d415ec70 2024-04-17T06:09:06.182 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1d3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1d4071020 0x7fc1d4148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:06.182 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1d3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1d4071020 0x7fc1d4148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44362/0 (socket says 172.21.15.135:44362) 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1d3fff640 1 -- 172.21.15.135:0/2721888216 learned_addr learned my addr 172.21.15.135:0/2721888216 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.181+0000 7fc1cbfff640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 0x7fc1d41f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d8aae640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d415ec70 0x7fc1d40cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d3fff640 1 -- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d415ec70 msgr2=0x7fc1d40cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d3fff640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d415ec70 0x7fc1d40cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d3fff640 1 -- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 msgr2=0x7fc1d41f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d3fff640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 0x7fc1d41f8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d3fff640 1 -- 172.21.15.135:0/2721888216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d41613f0 con 0x7fc1d4071020 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d8aae640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d415ec70 0x7fc1d40cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d3fff640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1d4071020 0x7fc1d4148250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4000c00 tx=0x7fc1c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d17fa640 1 -- 172.21.15.135:0/2721888216 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4035070 con 0x7fc1d4071020 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d17fa640 1 -- 172.21.15.135:0/2721888216 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4004230 con 0x7fc1d4071020 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d41f8dc0 con 0x7fc1d4071020 2024-04-17T06:09:06.183 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1d17fa640 1 -- 172.21.15.135:0/2721888216 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c403a630 con 0x7fc1d4071020 2024-04-17T06:09:06.184 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.182+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d4155170 con 0x7fc1d4071020 2024-04-17T06:09:06.184 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.183+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d40750f0 con 0x7fc1d4071020 2024-04-17T06:09:06.184 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.183+0000 7fc1d17fa640 1 -- 172.21.15.135:0/2721888216 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fc1c4038020 con 0x7fc1d4071020 2024-04-17T06:09:06.184 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.184+0000 7fc1d17fa640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc1b406c860 0x7fc1b406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:06.184 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.184+0000 7fc1cbfff640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc1b406c860 0x7fc1b406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:06.184 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.184+0000 7fc1d17fa640 1 -- 172.21.15.135:0/2721888216 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7fc1c40ac890 con 0x7fc1d4071020 2024-04-17T06:09:06.184 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.184+0000 7fc1cbfff640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc1b406c860 0x7fc1b406ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4154c20 tx=0x7fc1bc007d30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:06.186 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.189+0000 7fc1d17fa640 1 -- 172.21.15.135:0/2721888216 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1d40750f0 con 0x7fc1d4071020 2024-04-17T06:09:06.362 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.364+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1d41615c0 con 0x7fc1d4071020 2024-04-17T06:09:06.362 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.365+0000 7fc1d17fa640 1 -- 172.21.15.135:0/2721888216 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7fc1d41f8dc0 con 0x7fc1d4071020 2024-04-17T06:09:06.363 INFO:teuthology.misc.health.smithi135.stdout: 2024-04-17T06:09:06.363 INFO:teuthology.misc.health.smithi135.stdout:{"epoch":12,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:03.943306+0000","last_up_change":"0.000000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc1b406c860 msgr2=0x7fc1b406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc1b406c860 0x7fc1b406ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4154c20 tx=0x7fc1bc007d30 comp rx=0 tx=0).stop 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1d4071020 msgr2=0x7fc1d4148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1d4071020 0x7fc1d4148250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4000c00 tx=0x7fc1c40046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 shutdown_connections 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d415ec70 0x7fc1d40cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d415e890 0x7fc1d41f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc1b406c860 0x7fc1b406ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.365 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 --2- 172.21.15.135:0/2721888216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1d4071020 0x7fc1d4148250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:06.366 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 >> 172.21.15.135:0/2721888216 conn(0x7fc1d40730d0 msgr2=0x7fc1d40d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:06.366 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 shutdown_connections 2024-04-17T06:09:06.366 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:06.367+0000 7fc1da511640 1 -- 172.21.15.135:0/2721888216 wait complete. 2024-04-17T06:09:06.375 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-17T06:09:06.446 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-04-17T06:09:06.447+0000 7faf5fe3a640 -1 osd.7 0 waiting for initial osdmap 2024-04-17T06:09:06.456 INFO:tasks.ceph.osd.7.smithi190.stderr:2024-04-17T06:09:06.458+0000 7faf57c5a640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-17T06:09:13.181 DEBUG:teuthology.orchestra.run.smithi135:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:13.284 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.286+0000 7f26e7e7b640 1 Processor -- start 2024-04-17T06:09:13.284 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.286+0000 7f26e7e7b640 1 -- start start 2024-04-17T06:09:13.285 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.287+0000 7f26e7e7b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f26e015ebf0 0x7f26e00d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.285 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.287+0000 7f26e7e7b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 0x7f26e00b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.285 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.287+0000 7f26e7e7b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e00cca70 con 0x7f26e00b7850 2024-04-17T06:09:13.285 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.287+0000 7f26e7e7b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f26e00d0890 con 0x7f26e015ebf0 2024-04-17T06:09:13.285 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.287+0000 7f26e7e7b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f26e00d0b70 con 0x7f26e00d02d0 2024-04-17T06:09:13.286 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e5c17640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f26e015ebf0 0x7f26e00d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.286 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e5416640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 0x7f26e00b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.286 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e6418640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f26e00b7850 0x7f26e00b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39154/0 (socket says 172.21.15.135:39154) 2024-04-17T06:09:13.286 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e6418640 1 -- 172.21.15.135:0/756677153 learned_addr learned my addr 172.21.15.135:0/756677153 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:13.287 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e5c17640 1 -- 172.21.15.135:0/756677153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 msgr2=0x7f26e00b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:13.287 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e5c17640 1 --2- 172.21.15.135:0/756677153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 0x7f26e00b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.287 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e5c17640 1 -- 172.21.15.135:0/756677153 >> v1:172.21.15.190:6789/0 conn(0x7f26e00b7850 legacy=0x7f26e00b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.287 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e5c17640 1 -- 172.21.15.135:0/756677153 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e00d0e50 con 0x7f26e015ebf0 2024-04-17T06:09:13.287 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.288+0000 7f26e4c15640 1 -- 172.21.15.135:0/756677153 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955662490 0 0) 0x7f26e00cca70 con 0x7f26e00b7850 2024-04-17T06:09:13.288 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.289+0000 7f26e5c17640 1 --2- 172.21.15.135:0/756677153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e015ebf0 0x7f26e00d1930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0097a0 tx=0x7f26dc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=501f0a6b0b4f1cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.288 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.289+0000 7f26e4c15640 1 -- 172.21.15.135:0/756677153 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0315a0 con 0x7f26e015ebf0 2024-04-17T06:09:13.288 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.289+0000 7f26e4c15640 1 -- 172.21.15.135:0/756677153 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26dc037070 con 0x7f26e015ebf0 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.289+0000 7f26e4c15640 1 -- 172.21.15.135:0/756677153 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc031960 con 0x7f26e015ebf0 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.290+0000 7f26e7e7b640 1 -- 172.21.15.135:0/756677153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e015ebf0 msgr2=0x7f26e00d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.290+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/756677153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e015ebf0 0x7f26e00d1930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0097a0 tx=0x7f26dc02b840 comp rx=0 tx=0).stop 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.290+0000 7f26e7e7b640 1 -- 172.21.15.135:0/756677153 shutdown_connections 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.290+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/756677153 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 0x7f26e00b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.290+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/756677153 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e015ebf0 0x7f26e00d1930 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.290+0000 7f26e7e7b640 1 -- 172.21.15.135:0/756677153 >> 172.21.15.135:0/756677153 conn(0x7f26e00d2700 msgr2=0x7f26e00d2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.290+0000 7f26e7e7b640 1 -- 172.21.15.135:0/756677153 shutdown_connections 2024-04-17T06:09:13.289 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.290+0000 7f26e7e7b640 1 -- 172.21.15.135:0/756677153 wait complete. 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.291+0000 7f26e7e7b640 1 Processor -- start 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.291+0000 7f26e7e7b640 1 -- start start 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e7e7b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e00b7850 0x7f26e00c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e7e7b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 0x7f26e006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e7e7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26e015ebf0 0x7f26e006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e7e7b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26e00ccd50 con 0x7f26e015ebf0 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e7e7b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f26e00d1410 con 0x7f26e00b7850 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e7e7b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f26e00cced0 con 0x7f26e00d02d0 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e6418640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26e015ebf0 0x7f26e006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e6418640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26e015ebf0 0x7f26e006f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44392/0 (socket says 172.21.15.135:44392) 2024-04-17T06:09:13.293 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e6418640 1 -- 172.21.15.135:0/1028590579 learned_addr learned my addr 172.21.15.135:0/1028590579 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e5c17640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e00b7850 0x7f26e00c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e5416640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 0x7f26e006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e6418640 1 -- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 msgr2=0x7f26e006ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.292+0000 7f26e6418640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 0x7f26e006ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26e6418640 1 -- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e00b7850 msgr2=0x7f26e00c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26e6418640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e00b7850 0x7f26e00c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26e6418640 1 -- 172.21.15.135:0/1028590579 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e015aeb0 con 0x7f26e015ebf0 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26e5c17640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e00b7850 0x7f26e00c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:13.294 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26e6418640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26e015ebf0 0x7f26e006f8d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c810 tx=0x7f26d800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26d67fc640 1 -- 172.21.15.135:0/1028590579 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8018070 con 0x7f26e015ebf0 2024-04-17T06:09:13.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26d67fc640 1 -- 172.21.15.135:0/1028590579 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d800a640 con 0x7f26e015ebf0 2024-04-17T06:09:13.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e00c7860 con 0x7f26e015ebf0 2024-04-17T06:09:13.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26d67fc640 1 -- 172.21.15.135:0/1028590579 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d800aa30 con 0x7f26e015ebf0 2024-04-17T06:09:13.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.293+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e00c7e20 con 0x7f26e015ebf0 2024-04-17T06:09:13.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.294+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e004fd50 con 0x7f26e015ebf0 2024-04-17T06:09:13.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.294+0000 7f26d67fc640 1 -- 172.21.15.135:0/1028590579 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26d801b070 con 0x7f26e015ebf0 2024-04-17T06:09:13.295 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.295+0000 7f26d67fc640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f26c406c920 0x7f26c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.296 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.295+0000 7f26e5c17640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f26c406c920 0x7f26c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.296 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.295+0000 7f26d67fc640 1 -- 172.21.15.135:0/1028590579 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f26d808ae40 con 0x7f26e015ebf0 2024-04-17T06:09:13.296 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.296+0000 7f26e5c17640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f26c406c920 0x7f26c406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26dc036000 tx=0x7f26dc03d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.297 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.300+0000 7f26d67fc640 1 -- 172.21.15.135:0/1028590579 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f26d80595c0 con 0x7f26e015ebf0 2024-04-17T06:09:13.474 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.476+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26e0163240 con 0x7f26e015ebf0 2024-04-17T06:09:13.475 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.477+0000 7f26d67fc640 1 -- 172.21.15.135:0/1028590579 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f26d8058ec0 con 0x7f26e015ebf0 2024-04-17T06:09:13.476 INFO:teuthology.misc.health.smithi135.stdout: 2024-04-17T06:09:13.476 INFO:teuthology.misc.health.smithi135.stdout:{"epoch":16,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:09.962210+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:13.478 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.478+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f26c406c920 msgr2=0x7f26c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.478 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.478+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f26c406c920 0x7f26c406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f26dc036000 tx=0x7f26dc03d040 comp rx=0 tx=0).stop 2024-04-17T06:09:13.478 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.478+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26e015ebf0 msgr2=0x7f26e006f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.478 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.478+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26e015ebf0 0x7f26e006f8d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f26d800c810 tx=0x7f26d800ccd0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.479 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.479+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 shutdown_connections 2024-04-17T06:09:13.479 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.479+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f26c406c920 0x7f26c406ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.479 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.479+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26e015ebf0 0x7f26e006f8d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.479 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.479+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26e00d02d0 0x7f26e006ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.479 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.479+0000 7f26e7e7b640 1 --2- 172.21.15.135:0/1028590579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26e00b7850 0x7f26e00c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.479 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.479+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 >> 172.21.15.135:0/1028590579 conn(0x7f26e00d2700 msgr2=0x7f26e00b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:13.479 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.479+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 shutdown_connections 2024-04-17T06:09:13.480 INFO:teuthology.misc.health.smithi135.stderr:2024-04-17T06:09:13.479+0000 7f26e7e7b640 1 -- 172.21.15.135:0/1028590579 wait complete. 2024-04-17T06:09:13.486 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-17T06:09:13.486 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-17T06:09:13.487 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T06:09:13.487 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T06:09:13.489 INFO:tasks.ceph.mds.a:Started 2024-04-17T06:09:13.489 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T06:09:13.490 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T06:09:13.491 INFO:tasks.ceph.mds.c:Started 2024-04-17T06:09:13.492 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T06:09:13.492 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T06:09:13.495 INFO:tasks.ceph.mds.b:Started 2024-04-17T06:09:13.495 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T06:09:13.495 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T06:09:13.497 INFO:tasks.ceph.mds.d:Started 2024-04-17T06:09:13.498 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-17T06:09:13.498 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:13.567 INFO:tasks.ceph.mds.a.smithi135.stdout:starting mds.a at 2024-04-17T06:09:13.568 INFO:tasks.ceph.mds.c.smithi135.stdout:starting mds.c at 2024-04-17T06:09:13.585 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2024-04-17T06:09:13.587 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2024-04-17T06:09:13.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.609+0000 7f79b20b8640 1 Processor -- start 2024-04-17T06:09:13.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.609+0000 7f79b20b8640 1 -- start start 2024-04-17T06:09:13.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.609+0000 7f79b20b8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f79ac15ed00 0x7f79ac0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.609+0000 7f79b20b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac09bd50 0x7f79ac09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.609+0000 7f79b20b8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79ac0d13f0 con 0x7f79ac09af30 2024-04-17T06:09:13.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.609+0000 7f79b20b8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f79ac0d19b0 con 0x7f79ac15ed00 2024-04-17T06:09:13.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.609+0000 7f79b20b8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f79ac0d16d0 con 0x7f79ac09bd50 2024-04-17T06:09:13.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79ab7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f79ac15ed00 0x7f79ac0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79ab7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f79ac15ed00 0x7f79ac0d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44406/0 (socket says 172.21.15.135:44406) 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79ab7fe640 1 -- 172.21.15.135:0/3784536750 learned_addr learned my addr 172.21.15.135:0/3784536750 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79a3fff640 1 --2- 172.21.15.135:0/3784536750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac09bd50 0x7f79ac09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79ab7fe640 1 -- 172.21.15.135:0/3784536750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac09bd50 msgr2=0x7f79ac09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79ab7fe640 1 --2- 172.21.15.135:0/3784536750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac09bd50 0x7f79ac09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79ab7fe640 1 -- 172.21.15.135:0/3784536750 >> v1:172.21.15.190:6789/0 conn(0x7f79ac09af30 legacy=0x7f79ac0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79ab7fe640 1 -- 172.21.15.135:0/3784536750 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac06cc20 con 0x7f79ac15ed00 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.610+0000 7f79a3fff640 1 --2- 172.21.15.135:0/3784536750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac09bd50 0x7f79ac09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79ab7fe640 1 --2- 172.21.15.135:0/3784536750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac15ed00 0x7f79ac0d0ba0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f799c009870 tx=0x7f799c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32dc3f77e8257828 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79aaffd640 1 -- 172.21.15.135:0/3784536750 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c035020 con 0x7f79ac15ed00 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79aaffd640 1 -- 172.21.15.135:0/3784536750 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c031640 con 0x7f79ac15ed00 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79aaffd640 1 -- 172.21.15.135:0/3784536750 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c035020 con 0x7f79ac15ed00 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79b20b8640 1 -- 172.21.15.135:0/3784536750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac15ed00 msgr2=0x7f79ac0d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79b20b8640 1 --2- 172.21.15.135:0/3784536750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac15ed00 0x7f79ac0d0ba0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f799c009870 tx=0x7f799c02b8f0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79b20b8640 1 -- 172.21.15.135:0/3784536750 shutdown_connections 2024-04-17T06:09:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79b20b8640 1 --2- 172.21.15.135:0/3784536750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac09bd50 0x7f79ac09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79b20b8640 1 --2- 172.21.15.135:0/3784536750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac15ed00 0x7f79ac0d0ba0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.611+0000 7f79b20b8640 1 -- 172.21.15.135:0/3784536750 >> 172.21.15.135:0/3784536750 conn(0x7f79ac0b4190 msgr2=0x7f79ac0b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.612+0000 7f79b20b8640 1 -- 172.21.15.135:0/3784536750 shutdown_connections 2024-04-17T06:09:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.612+0000 7f79b20b8640 1 -- 172.21.15.135:0/3784536750 wait complete. 2024-04-17T06:09:13.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.613+0000 7f79b20b8640 1 Processor -- start 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.613+0000 7f79b20b8640 1 -- start start 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.613+0000 7f79b20b8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ac09af30 0x7f79ac15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.613+0000 7f79b20b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac09bd50 0x7f79ac158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.613+0000 7f79b20b8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac15ed00 0x7f79ac0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.613+0000 7f79b20b8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac06e430 con 0x7f79ac09bd50 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.613+0000 7f79b20b8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac0d10e0 con 0x7f79ac09af30 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.613+0000 7f79b20b8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f79ac06e5b0 con 0x7f79ac15ed00 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac09bd50 0x7f79ac158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac09bd50 0x7f79ac158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44420/0 (socket says 172.21.15.135:44420) 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 -- 172.21.15.135:0/2835263887 learned_addr learned my addr 172.21.15.135:0/2835263887 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79ab7fe640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ac09af30 0x7f79ac15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79abfff640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac15ed00 0x7f79ac0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 -- 172.21.15.135:0/2835263887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac15ed00 msgr2=0x7f79ac0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac15ed00 0x7f79ac0cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 -- 172.21.15.135:0/2835263887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ac09af30 msgr2=0x7f79ac15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ac09af30 0x7f79ac15b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 -- 172.21.15.135:0/2835263887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac0caa10 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a3fff640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac09bd50 0x7f79ac158d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f799400c7f0 tx=0x7f799400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a8ff9640 1 -- 172.21.15.135:0/2835263887 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994015070 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a8ff9640 1 -- 172.21.15.135:0/2835263887 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7994007820 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79a8ff9640 1 -- 172.21.15.135:0/2835263887 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994007bd0 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ac0cacf0 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.614+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ac0cb2b0 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.615+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ac068a20 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.616+0000 7f79a8ff9640 1 -- 172.21.15.135:0/2835263887 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7994018070 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.616+0000 7f79a8ff9640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f798c06c8f0 0x7f798c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.616+0000 7f79a8ff9640 1 -- 172.21.15.135:0/2835263887 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f799408b7d0 con 0x7f79ac09bd50 2024-04-17T06:09:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.616+0000 7f79ab7fe640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f798c06c8f0 0x7f798c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.616+0000 7f79ab7fe640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f798c06c8f0 0x7f798c06edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f799c036000 tx=0x7f799c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.621+0000 7f79a8ff9640 1 -- 172.21.15.135:0/2835263887 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7994059990 con 0x7f79ac09bd50 2024-04-17T06:09:13.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.798+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79ac0694e0 con 0x7f79ac09bd50 2024-04-17T06:09:13.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.799+0000 7f79a8ff9640 1 -- 172.21.15.135:0/2835263887 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f799405ee20 con 0x7f79ac09bd50 2024-04-17T06:09:13.797 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:13.797 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":16,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:09.962210+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:13.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f798c06c8f0 msgr2=0x7f798c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f798c06c8f0 0x7f798c06edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f799c036000 tx=0x7f799c03c040 comp rx=0 tx=0).stop 2024-04-17T06:09:13.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac09bd50 msgr2=0x7f79ac158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac09bd50 0x7f79ac158d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f799400c7f0 tx=0x7f799400ccb0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 shutdown_connections 2024-04-17T06:09:13.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79ac15ed00 0x7f79ac0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f798c06c8f0 0x7f798c06edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f79ac09bd50 0x7f79ac158d30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 --2- 172.21.15.135:0/2835263887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79ac09af30 0x7f79ac15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 >> 172.21.15.135:0/2835263887 conn(0x7f79ac0b4190 msgr2=0x7f79ac0717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:13.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 shutdown_connections 2024-04-17T06:09:13.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.802+0000 7f79b20b8640 1 -- 172.21.15.135:0/2835263887 wait complete. 2024-04-17T06:09:13.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:09:13.811 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.920+0000 7f12886b1640 1 Processor -- start 2024-04-17T06:09:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.921+0000 7f12886b1640 1 -- start start 2024-04-17T06:09:13.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.921+0000 7f12886b1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f128009bd50 0x7f128009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.921+0000 7f12886b1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009af30 0x7f128006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.921+0000 7f12886b1640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12800d13f0 con 0x7f128015ed00 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.921+0000 7f12886b1640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f12800d19b0 con 0x7f128009bd50 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.921+0000 7f12886b1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f12800d16d0 con 0x7f128009af30 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f1285c4c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f128009bd50 0x7f128009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f128644d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f128015ed00 0x7f12800d0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:59792/0 (socket says 172.21.15.135:59792) 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f128644d640 1 -- 172.21.15.135:0/2683172940 learned_addr learned my addr 172.21.15.135:0/2683172940 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f1286c4e640 1 --2- 172.21.15.135:0/2683172940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009af30 0x7f128006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f1285c4c640 1 -- 172.21.15.135:0/2683172940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009af30 msgr2=0x7f128006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f1285c4c640 1 --2- 172.21.15.135:0/2683172940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009af30 0x7f128006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f1285c4c640 1 -- 172.21.15.135:0/2683172940 >> v1:172.21.15.135:6789/0 conn(0x7f128015ed00 legacy=0x7f12800d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f1285c4c640 1 -- 172.21.15.135:0/2683172940 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128006cc20 con 0x7f128009bd50 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.922+0000 7f128544b640 1 -- 172.21.15.135:0/2683172940 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432628590 0 0) 0x7f12800d13f0 con 0x7f128015ed00 2024-04-17T06:09:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f1285c4c640 1 --2- 172.21.15.135:0/2683172940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128009bd50 0x7f128009a9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f127c0097a0 tx=0x7f127c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc5394b9354e285 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f128544b640 1 -- 172.21.15.135:0/2683172940 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c035020 con 0x7f128009bd50 2024-04-17T06:09:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f128544b640 1 -- 172.21.15.135:0/2683172940 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c031830 con 0x7f128009bd50 2024-04-17T06:09:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f128544b640 1 -- 172.21.15.135:0/2683172940 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c031c00 con 0x7f128009bd50 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f12886b1640 1 -- 172.21.15.135:0/2683172940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128009bd50 msgr2=0x7f128009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f12886b1640 1 --2- 172.21.15.135:0/2683172940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128009bd50 0x7f128009a9f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f127c0097a0 tx=0x7f127c02b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f12886b1640 1 -- 172.21.15.135:0/2683172940 shutdown_connections 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f12886b1640 1 --2- 172.21.15.135:0/2683172940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009af30 0x7f128006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f12886b1640 1 --2- 172.21.15.135:0/2683172940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128009bd50 0x7f128009a9f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f12886b1640 1 -- 172.21.15.135:0/2683172940 >> 172.21.15.135:0/2683172940 conn(0x7f12800b4190 msgr2=0x7f12800b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f12886b1640 1 -- 172.21.15.135:0/2683172940 shutdown_connections 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.923+0000 7f12886b1640 1 -- 172.21.15.135:0/2683172940 wait complete. 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f12886b1640 1 Processor -- start 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f12886b1640 1 -- start start 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f12886b1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f128009af30 0x7f1280151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f12886b1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009bd50 0x7f12800cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f12886b1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128015ed00 0x7f12800cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f12886b1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f128006dae0 con 0x7f128015ed00 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f12886b1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f128006d960 con 0x7f128009af30 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f12886b1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f128006dc60 con 0x7f128009bd50 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f1286c4e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128015ed00 0x7f12800cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f1286c4e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128015ed00 0x7f12800cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44432/0 (socket says 172.21.15.135:44432) 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.925+0000 7f1286c4e640 1 -- 172.21.15.135:0/3268547452 learned_addr learned my addr 172.21.15.135:0/3268547452 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1285c4c640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009bd50 0x7f12800cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f128644d640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f128009af30 0x7f1280151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1286c4e640 1 -- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009bd50 msgr2=0x7f12800cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1286c4e640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009bd50 0x7f12800cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1286c4e640 1 -- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f128009af30 msgr2=0x7f1280151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:13.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1286c4e640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f128009af30 0x7f1280151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1286c4e640 1 -- 172.21.15.135:0/3268547452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1280152200 con 0x7f128015ed00 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1285c4c640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009bd50 0x7f12800cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f128644d640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f128009af30 0x7f1280151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1286c4e640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128015ed00 0x7f12800cb5d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1278007e70 tx=0x7f127800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1272ffd640 1 -- 172.21.15.135:0/3268547452 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278014620 con 0x7f128015ed00 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1272ffd640 1 -- 172.21.15.135:0/3268547452 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1278019070 con 0x7f128015ed00 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128015fef0 con 0x7f128015ed00 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f1272ffd640 1 -- 172.21.15.135:0/3268547452 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278014a20 con 0x7f128015ed00 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.926+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12801604b0 con 0x7f128015ed00 2024-04-17T06:09:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.927+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1280068a20 con 0x7f128015ed00 2024-04-17T06:09:13.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.928+0000 7f1272ffd640 1 -- 172.21.15.135:0/3268547452 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1278014bc0 con 0x7f128015ed00 2024-04-17T06:09:13.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.928+0000 7f1272ffd640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f126406c8f0 0x7f126406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:13.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.928+0000 7f1272ffd640 1 -- 172.21.15.135:0/3268547452 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f127808c110 con 0x7f128015ed00 2024-04-17T06:09:13.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.928+0000 7f128644d640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f126406c8f0 0x7f126406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:13.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.929+0000 7f128644d640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f126406c8f0 0x7f126406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1274000f60 tx=0x7f1274009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:13.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:13.932+0000 7f1272ffd640 1 -- 172.21.15.135:0/3268547452 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1278011260 con 0x7f128015ed00 2024-04-17T06:09:14.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.106+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f128006d2e0 con 0x7f128015ed00 2024-04-17T06:09:14.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.107+0000 7f1272ffd640 1 -- 172.21.15.135:0/3268547452 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f127805a890 con 0x7f128015ed00 2024-04-17T06:09:14.105 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f126406c8f0 msgr2=0x7f126406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f126406c8f0 0x7f126406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1274000f60 tx=0x7f1274009040 comp rx=0 tx=0).stop 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128015ed00 msgr2=0x7f12800cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128015ed00 0x7f12800cb5d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1278007e70 tx=0x7f127800e4b0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 shutdown_connections 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f126406c8f0 0x7f126406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f128015ed00 0x7f12800cb5d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f128009bd50 0x7f12800cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.108+0000 7f12886b1640 1 --2- 172.21.15.135:0/3268547452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f128009af30 0x7f1280151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.109+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 >> 172.21.15.135:0/3268547452 conn(0x7f12800b4190 msgr2=0x7f12800706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.109+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 shutdown_connections 2024-04-17T06:09:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.109+0000 7f12886b1640 1 -- 172.21.15.135:0/3268547452 wait complete. 2024-04-17T06:09:14.117 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:09:14.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.222+0000 7ff75be93640 1 Processor -- start 2024-04-17T06:09:14.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.223+0000 7ff75be93640 1 -- start start 2024-04-17T06:09:14.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.223+0000 7ff75be93640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7540cd1a0 0x7ff7540cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff75be93640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75415d9b0 0x7ff7540cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff75be93640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75406ba30 con 0x7ff7540b3800 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff75be93640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff75406c600 con 0x7ff7540cd1a0 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff75be93640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff7540cdac0 con 0x7ff75415d9b0 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff759c2f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7540cd1a0 0x7ff7540cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff759c2f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff7540cd1a0 0x7ff7540cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44440/0 (socket says 172.21.15.135:44440) 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff759c2f640 1 -- 172.21.15.135:0/1911391026 learned_addr learned my addr 172.21.15.135:0/1911391026 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff75942e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff7540b3800 0x7ff7540cf150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:59808/0 (socket says 172.21.15.135:59808) 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff75a430640 1 --2- 172.21.15.135:0/1911391026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75415d9b0 0x7ff7540cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff759c2f640 1 -- 172.21.15.135:0/1911391026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75415d9b0 msgr2=0x7ff7540cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff759c2f640 1 --2- 172.21.15.135:0/1911391026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75415d9b0 0x7ff7540cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff759c2f640 1 -- 172.21.15.135:0/1911391026 >> v1:172.21.15.135:6789/0 conn(0x7ff7540b3800 legacy=0x7ff7540cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff759c2f640 1 -- 172.21.15.135:0/1911391026 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75406bcd0 con 0x7ff7540cd1a0 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.224+0000 7ff758c2d640 1 -- 172.21.15.135:0/1911391026 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522288413 0 0) 0x7ff75406ba30 con 0x7ff7540b3800 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff759c2f640 1 --2- 172.21.15.135:0/1911391026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7540cd1a0 0x7ff7540cd580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009870 tx=0x7ff74c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aaf97539133e2d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff758c2d640 1 -- 172.21.15.135:0/1911391026 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c035020 con 0x7ff7540cd1a0 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff758c2d640 1 -- 172.21.15.135:0/1911391026 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74c031640 con 0x7ff7540cd1a0 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff758c2d640 1 -- 172.21.15.135:0/1911391026 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c02d040 con 0x7ff7540cd1a0 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff75be93640 1 -- 172.21.15.135:0/1911391026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7540cd1a0 msgr2=0x7ff7540cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff75be93640 1 --2- 172.21.15.135:0/1911391026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7540cd1a0 0x7ff7540cd580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009870 tx=0x7ff74c02b8f0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff75be93640 1 -- 172.21.15.135:0/1911391026 shutdown_connections 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff75be93640 1 --2- 172.21.15.135:0/1911391026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff75415d9b0 0x7ff7540cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff75be93640 1 --2- 172.21.15.135:0/1911391026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7540cd1a0 0x7ff7540cd580 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.225+0000 7ff75be93640 1 -- 172.21.15.135:0/1911391026 >> 172.21.15.135:0/1911391026 conn(0x7ff75409a9f0 msgr2=0x7ff7540b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:14.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.226+0000 7ff75be93640 1 -- 172.21.15.135:0/1911391026 shutdown_connections 2024-04-17T06:09:14.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.226+0000 7ff75be93640 1 -- 172.21.15.135:0/1911391026 wait complete. 2024-04-17T06:09:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.227+0000 7ff75be93640 1 Processor -- start 2024-04-17T06:09:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.227+0000 7ff75be93640 1 -- start start 2024-04-17T06:09:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.227+0000 7ff75be93640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7540b3800 0x7ff7540c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.227+0000 7ff75be93640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7540cd1a0 0x7ff7541508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.227+0000 7ff75be93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75415d9b0 0x7ff7540c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.227+0000 7ff75be93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7540ced60 con 0x7ff75415d9b0 2024-04-17T06:09:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.227+0000 7ff75be93640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff75406c290 con 0x7ff7540b3800 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.227+0000 7ff75be93640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff7540d0370 con 0x7ff7540cd1a0 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75415d9b0 0x7ff7540c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75415d9b0 0x7ff7540c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:44452/0 (socket says 172.21.15.135:44452) 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 -- 172.21.15.135:0/1138626952 learned_addr learned my addr 172.21.15.135:0/1138626952 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75942e640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7540cd1a0 0x7ff7541508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff759c2f640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7540b3800 0x7ff7540c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 -- 172.21.15.135:0/1138626952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7540cd1a0 msgr2=0x7ff7541508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7540cd1a0 0x7ff7541508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 -- 172.21.15.135:0/1138626952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7540b3800 msgr2=0x7ff7540c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7540b3800 0x7ff7540c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 -- 172.21.15.135:0/1138626952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75415b0b0 con 0x7ff75415d9b0 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75a430640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75415d9b0 0x7ff7540c9210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff74400d800 tx=0x7ff74400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff7527fc640 1 -- 172.21.15.135:0/1138626952 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74400b6c0 con 0x7ff75415d9b0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff7527fc640 1 -- 172.21.15.135:0/1138626952 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff744019070 con 0x7ff75415d9b0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7540c6b90 con 0x7ff75415d9b0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff7527fc640 1 -- 172.21.15.135:0/1138626952 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74400bac0 con 0x7ff75415d9b0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.228+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7540c6e00 con 0x7ff75415d9b0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.229+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff754074940 con 0x7ff75415d9b0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.230+0000 7ff7527fc640 1 -- 172.21.15.135:0/1138626952 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff74400bc60 con 0x7ff75415d9b0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.230+0000 7ff7527fc640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff74006c8f0 0x7ff74006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.230+0000 7ff759c2f640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff74006c8f0 0x7ff74006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.230+0000 7ff7527fc640 1 -- 172.21.15.135:0/1138626952 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff74408bb30 con 0x7ff75415d9b0 2024-04-17T06:09:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.230+0000 7ff759c2f640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff74006c8f0 0x7ff74006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff74c036000 tx=0x7ff74c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:14.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.234+0000 7ff7527fc640 1 -- 172.21.15.135:0/1138626952 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff744059c60 con 0x7ff75415d9b0 2024-04-17T06:09:14.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.446+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7540cf700 con 0x7ff75415d9b0 2024-04-17T06:09:14.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.446+0000 7ff7527fc640 1 -- 172.21.15.135:0/1138626952 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7ff74405df60 con 0x7ff75415d9b0 2024-04-17T06:09:14.444 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:14.444 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-17T06:09:14.445 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 2 2024-04-17T06:09:14.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff74006c8f0 msgr2=0x7ff74006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff74006c8f0 0x7ff74006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff74c036000 tx=0x7ff74c03c040 comp rx=0 tx=0).stop 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75415d9b0 msgr2=0x7ff7540c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75415d9b0 0x7ff7540c9210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff74400d800 tx=0x7ff74400dcc0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 shutdown_connections 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff74006c8f0 0x7ff74006edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff75415d9b0 0x7ff7540c9210 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff7540cd1a0 0x7ff7541508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 --2- 172.21.15.135:0/1138626952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff7540b3800 0x7ff7540c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:14.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 >> 172.21.15.135:0/1138626952 conn(0x7ff75409a9f0 msgr2=0x7ff75415f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:14.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 shutdown_connections 2024-04-17T06:09:14.448 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:14.448+0000 7ff75be93640 1 -- 172.21.15.135:0/1138626952 wait complete. 2024-04-17T06:09:16.459 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:09:16.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.577+0000 7f5983d90640 1 Processor -- start 2024-04-17T06:09:16.575 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.577+0000 7f5983d90640 1 -- start start 2024-04-17T06:09:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f5983d90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c09bbe0 0x7f597c09b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.576 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f5983d90640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f597c0cd630 0x7f597c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f5983d90640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f597c06db80 con 0x7f597c15ee30 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f5983d90640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f597c06de60 con 0x7f597c0cd630 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f5983d90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f597c0cce30 con 0x7f597c09bbe0 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f598232d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f597c0cd630 0x7f597c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f598232d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f597c0cd630 0x7f597c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53616/0 (socket says 172.21.15.135:53616) 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f598232d640 1 -- 172.21.15.135:0/5847423 learned_addr learned my addr 172.21.15.135:0/5847423 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f598132b640 1 --2- 172.21.15.135:0/5847423 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c09bbe0 0x7f597c09b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f598232d640 1 -- 172.21.15.135:0/5847423 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c09bbe0 msgr2=0x7f597c09b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f598232d640 1 --2- 172.21.15.135:0/5847423 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c09bbe0 0x7f597c09b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f598232d640 1 -- 172.21.15.135:0/5847423 >> v1:172.21.15.190:6789/0 conn(0x7f597c15ee30 legacy=0x7f597c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:16.577 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.578+0000 7f598232d640 1 -- 172.21.15.135:0/5847423 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0cd110 con 0x7f597c0cd630 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.579+0000 7f598232d640 1 --2- 172.21.15.135:0/5847423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0cda80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5974009e50 tx=0x7f597402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689ee1388b2eb764 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.579+0000 7f5980b2a640 1 -- 172.21.15.135:0/5847423 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597402ca70 con 0x7f597c0cd630 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.579+0000 7f5980b2a640 1 -- 172.21.15.135:0/5847423 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597402cc10 con 0x7f597c0cd630 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.579+0000 7f5980b2a640 1 -- 172.21.15.135:0/5847423 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59740343f0 con 0x7f597c0cd630 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.580+0000 7f5983d90640 1 -- 172.21.15.135:0/5847423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 msgr2=0x7f597c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.580+0000 7f5983d90640 1 --2- 172.21.15.135:0/5847423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0cda80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5974009e50 tx=0x7f597402bed0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.580+0000 7f5983d90640 1 -- 172.21.15.135:0/5847423 shutdown_connections 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.580+0000 7f5983d90640 1 --2- 172.21.15.135:0/5847423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0cda80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.580+0000 7f5983d90640 1 --2- 172.21.15.135:0/5847423 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c09bbe0 0x7f597c09b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.580+0000 7f5983d90640 1 -- 172.21.15.135:0/5847423 >> 172.21.15.135:0/5847423 conn(0x7f597c0b41a0 msgr2=0x7f597c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:16.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.580+0000 7f5983d90640 1 -- 172.21.15.135:0/5847423 shutdown_connections 2024-04-17T06:09:16.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.580+0000 7f5983d90640 1 -- 172.21.15.135:0/5847423 wait complete. 2024-04-17T06:09:16.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.581+0000 7f5983d90640 1 Processor -- start 2024-04-17T06:09:16.580 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.581+0000 7f5983d90640 1 -- start start 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f5983d90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c09bbe0 0x7f597c1f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f5983d90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f5983d90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c15ee30 0x7f597c0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f5983d90640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f597c06d910 con 0x7f597c0cd630 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f5983d90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f597c06d790 con 0x7f597c09bbe0 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f5983d90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f597c09b640 con 0x7f597c15ee30 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53632/0 (socket says 172.21.15.135:53632) 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 -- 172.21.15.135:0/2543308546 learned_addr learned my addr 172.21.15.135:0/2543308546 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:16.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598232d640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c15ee30 0x7f597c0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f5981b2c640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c09bbe0 0x7f597c1f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 -- 172.21.15.135:0/2543308546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c15ee30 msgr2=0x7f597c0c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c15ee30 0x7f597c0c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 -- 172.21.15.135:0/2543308546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c09bbe0 msgr2=0x7f597c1f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c09bbe0 0x7f597c1f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 -- 172.21.15.135:0/2543308546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c0c6720 con 0x7f597c0cd630 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f598132b640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0c61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f597800d800 tx=0x7f597800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.582+0000 7f59727fc640 1 -- 172.21.15.135:0/2543308546 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597800b6c0 con 0x7f597c0cd630 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.583+0000 7f59727fc640 1 -- 172.21.15.135:0/2543308546 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5978019070 con 0x7f597c0cd630 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.583+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c0c69c0 con 0x7f597c0cd630 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.583+0000 7f59727fc640 1 -- 172.21.15.135:0/2543308546 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597800ba80 con 0x7f597c0cd630 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.583+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c0c6f80 con 0x7f597c0cd630 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.584+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c074940 con 0x7f597c0cd630 2024-04-17T06:09:16.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.584+0000 7f59727fc640 1 -- 172.21.15.135:0/2543308546 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f597800bc20 con 0x7f597c0cd630 2024-04-17T06:09:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.585+0000 7f59727fc640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f596006c8f0 0x7f596006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.585+0000 7f5981b2c640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f596006c8f0 0x7f596006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.585+0000 7f59727fc640 1 -- 172.21.15.135:0/2543308546 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f597808bae0 con 0x7f597c0cd630 2024-04-17T06:09:16.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.585+0000 7f5981b2c640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f596006c8f0 0x7f596006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f596c006fd0 tx=0x7f596c009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:16.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.589+0000 7f59727fc640 1 -- 172.21.15.135:0/2543308546 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5978059ca0 con 0x7f597c0cd630 2024-04-17T06:09:16.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.800+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f597c0c7640 con 0x7f597c0cd630 2024-04-17T06:09:16.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.800+0000 7f59727fc640 1 -- 172.21.15.135:0/2543308546 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f597805dfa0 con 0x7f597c0cd630 2024-04-17T06:09:16.798 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:16.798 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-17T06:09:16.799 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 3 2024-04-17T06:09:16.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f596006c8f0 msgr2=0x7f596006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f596006c8f0 0x7f596006edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f596c006fd0 tx=0x7f596c009060 comp rx=0 tx=0).stop 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 msgr2=0x7f597c0c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0c61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f597800d800 tx=0x7f597800dcc0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 shutdown_connections 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c15ee30 0x7f597c0c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f596006c8f0 0x7f596006edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f597c0cd630 0x7f597c0c61e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 --2- 172.21.15.135:0/2543308546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c09bbe0 0x7f597c1f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 >> 172.21.15.135:0/2543308546 conn(0x7f597c0b41a0 msgr2=0x7f597c0d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 shutdown_connections 2024-04-17T06:09:16.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.802+0000 7f5983d90640 1 -- 172.21.15.135:0/2543308546 wait complete. 2024-04-17T06:09:16.811 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:16.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.916+0000 7ff58553d640 1 Processor -- start 2024-04-17T06:09:16.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.916+0000 7ff58553d640 1 -- start start 2024-04-17T06:09:16.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff58553d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5800d43d0 0x7ff5800d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff58553d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5800d30a0 con 0x7ff58015ec00 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff58553d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5800d3330 con 0x7ff5800d1810 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff58553d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff58015efe0 con 0x7ff5800d43d0 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff57e7fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5800d43d0 0x7ff5800d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff57e7fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5800d43d0 0x7ff5800d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53640/0 (socket says 172.21.15.135:53640) 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff57e7fc640 1 -- 172.21.15.135:0/1837821510 learned_addr learned my addr 172.21.15.135:0/1837821510 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff57effd640 1 --1- 172.21.15.135:0/1837821510 >> v1:172.21.15.135:6789/0 conn(0x7ff58015ec00 0x7ff5800d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:48996/0 (socket says 172.21.15.135:48996) 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff57e7fc640 1 -- 172.21.15.135:0/1837821510 >> v1:172.21.15.190:6789/0 conn(0x7ff5800d1810 legacy=0x7ff5800b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.917+0000 7ff57e7fc640 1 -- 172.21.15.135:0/1837821510 >> v1:172.21.15.135:6789/0 conn(0x7ff58015ec00 legacy=0x7ff5800d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.918+0000 7ff57e7fc640 1 -- 172.21.15.135:0/1837821510 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5800d3610 con 0x7ff5800d43d0 2024-04-17T06:09:16.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.918+0000 7ff57dffb640 1 -- 172.21.15.135:0/1837821510 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659884244 0 0) 0x7ff5800d30a0 con 0x7ff58015ec00 2024-04-17T06:09:16.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.918+0000 7ff57e7fc640 1 --2- 172.21.15.135:0/1837821510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff5800d12d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff574009870 tx=0x7ff57402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a44caa48bb7dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.918+0000 7ff57dffb640 1 -- 172.21.15.135:0/1837821510 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57402f840 con 0x7ff5800d43d0 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.918+0000 7ff57dffb640 1 -- 172.21.15.135:0/1837821510 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57402f9e0 con 0x7ff5800d43d0 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.918+0000 7ff57dffb640 1 -- 172.21.15.135:0/1837821510 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57402fdb0 con 0x7ff5800d43d0 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.919+0000 7ff58553d640 1 -- 172.21.15.135:0/1837821510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 msgr2=0x7ff5800d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.919+0000 7ff58553d640 1 --2- 172.21.15.135:0/1837821510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff5800d12d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff574009870 tx=0x7ff57402ba20 comp rx=0 tx=0).stop 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.919+0000 7ff58553d640 1 -- 172.21.15.135:0/1837821510 shutdown_connections 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.919+0000 7ff58553d640 1 --2- 172.21.15.135:0/1837821510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff5800d12d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.919+0000 7ff58553d640 1 -- 172.21.15.135:0/1837821510 >> 172.21.15.135:0/1837821510 conn(0x7ff58006cc90 msgr2=0x7ff58006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.919+0000 7ff58553d640 1 -- 172.21.15.135:0/1837821510 shutdown_connections 2024-04-17T06:09:16.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.919+0000 7ff58553d640 1 -- 172.21.15.135:0/1837821510 wait complete. 2024-04-17T06:09:16.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.920+0000 7ff58553d640 1 Processor -- start 2024-04-17T06:09:16.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.920+0000 7ff58553d640 1 -- start start 2024-04-17T06:09:16.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff58553d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5800d1810 0x7ff58015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff58553d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff580150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff58553d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58015ec00 0x7ff5800c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff58553d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800d1c60 con 0x7ff5800d43d0 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff58553d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800d3f40 con 0x7ff58015ec00 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff58553d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff5800d3bd0 con 0x7ff5800d1810 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff580150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff580150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53652/0 (socket says 172.21.15.135:53652) 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57e7fc640 1 -- 172.21.15.135:0/1480264422 learned_addr learned my addr 172.21.15.135:0/1480264422 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57effd640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5800d1810 0x7ff58015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57f7fe640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58015ec00 0x7ff5800c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57e7fc640 1 -- 172.21.15.135:0/1480264422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5800d1810 msgr2=0x7ff58015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57e7fc640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5800d1810 0x7ff58015fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57e7fc640 1 -- 172.21.15.135:0/1480264422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58015ec00 msgr2=0x7ff5800c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57e7fc640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58015ec00 0x7ff5800c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:16.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.921+0000 7ff57e7fc640 1 -- 172.21.15.135:0/1480264422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5800c8820 con 0x7ff5800d43d0 2024-04-17T06:09:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.922+0000 7ff57e7fc640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff580150f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff574000c00 tx=0x7ff574004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.922+0000 7ff5677fe640 1 -- 172.21.15.135:0/1480264422 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57402f800 con 0x7ff5800d43d0 2024-04-17T06:09:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.922+0000 7ff5677fe640 1 -- 172.21.15.135:0/1480264422 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff574046070 con 0x7ff5800d43d0 2024-04-17T06:09:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.922+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5800c8b00 con 0x7ff5800d43d0 2024-04-17T06:09:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.922+0000 7ff5677fe640 1 -- 172.21.15.135:0/1480264422 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574041640 con 0x7ff5800d43d0 2024-04-17T06:09:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.922+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5800c90c0 con 0x7ff5800d43d0 2024-04-17T06:09:16.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.923+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff580074940 con 0x7ff5800d43d0 2024-04-17T06:09:16.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.923+0000 7ff5677fe640 1 -- 172.21.15.135:0/1480264422 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff574040070 con 0x7ff5800d43d0 2024-04-17T06:09:16.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.924+0000 7ff5677fe640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:16.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.924+0000 7ff5677fe640 1 -- 172.21.15.135:0/1480264422 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff5740ae010 con 0x7ff5800d43d0 2024-04-17T06:09:16.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.924+0000 7ff57effd640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:16.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.924+0000 7ff57effd640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff56c06c8f0 0x7ff56c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff570000f60 tx=0x7ff570009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:16.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:16.928+0000 7ff5677fe640 1 -- 172.21.15.135:0/1480264422 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff574033030 con 0x7ff5800d43d0 2024-04-17T06:09:17.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.104+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff58006bd20 con 0x7ff5800d43d0 2024-04-17T06:09:17.102 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.104+0000 7ff5677fe640 1 -- 172.21.15.135:0/1480264422 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7ff57407c790 con 0x7ff5800d43d0 2024-04-17T06:09:17.102 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:17.103 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":16,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:09.962210+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff56c06c8f0 msgr2=0x7ff56c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff56c06c8f0 0x7ff56c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff570000f60 tx=0x7ff570009040 comp rx=0 tx=0).stop 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 msgr2=0x7ff580150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff580150f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff574000c00 tx=0x7ff574004730 comp rx=0 tx=0).stop 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 shutdown_connections 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff58015ec00 0x7ff5800c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5800d43d0 0x7ff580150f00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 --2- 172.21.15.135:0/1480264422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5800d1810 0x7ff58015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 >> 172.21.15.135:0/1480264422 conn(0x7ff58006cc90 msgr2=0x7ff5800b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 shutdown_connections 2024-04-17T06:09:17.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.106+0000 7ff58553d640 1 -- 172.21.15.135:0/1480264422 wait complete. 2024-04-17T06:09:17.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:09:17.116 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.224+0000 7fbcd81e8640 1 Processor -- start 2024-04-17T06:09:17.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.224+0000 7fbcd81e8640 1 -- start start 2024-04-17T06:09:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd81e8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbcd00ba560 0x7fbcd00d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd81e8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd00d3230 con 0x7fbcd015ebb0 2024-04-17T06:09:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd81e8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd00d3510 con 0x7fbcd00d1850 2024-04-17T06:09:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd81e8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd00d37f0 con 0x7fbcd00ba560 2024-04-17T06:09:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd5f84640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbcd015ebb0 0x7fbcd00d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:49006/0 (socket says 172.21.15.135:49006) 2024-04-17T06:09:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd5f84640 1 -- 172.21.15.135:0/2713491773 learned_addr learned my addr 172.21.15.135:0/2713491773 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:17.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd5783640 1 --2- 172.21.15.135:0/2713491773 >> v2:172.21.15.190:3300/0 conn(0x7fbcd00ba560 0x7fbcd00d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd4f82640 1 -- 172.21.15.135:0/2713491773 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521546987 0 0) 0x7fbcd00d3230 con 0x7fbcd015ebb0 2024-04-17T06:09:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd4f82640 1 -- 172.21.15.135:0/2713491773 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcbc003520 con 0x7fbcd015ebb0 2024-04-17T06:09:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd5783640 1 -- 172.21.15.135:0/2713491773 >> v1:172.21.15.190:6789/0 conn(0x7fbcd00d1850 legacy=0x7fbcd00b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd4f82640 1 -- 172.21.15.135:0/2713491773 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2944015430 0 0) 0x7fbcbc003520 con 0x7fbcd015ebb0 2024-04-17T06:09:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd5783640 1 -- 172.21.15.135:0/2713491773 >> v1:172.21.15.135:6789/0 conn(0x7fbcd015ebb0 legacy=0x7fbcd00d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.225+0000 7fbcd5783640 1 -- 172.21.15.135:0/2713491773 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd00d3ad0 con 0x7fbcd00ba560 2024-04-17T06:09:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.226+0000 7fbcd5783640 1 --2- 172.21.15.135:0/2713491773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd00ba560 0x7fbcd00d42d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00097a0 tx=0x7fbcc002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff11bfbdd04cfbb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.226+0000 7fbcd4f82640 1 -- 172.21.15.135:0/2713491773 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0038070 con 0x7fbcd00ba560 2024-04-17T06:09:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.226+0000 7fbcd4f82640 1 -- 172.21.15.135:0/2713491773 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc0031500 con 0x7fbcd00ba560 2024-04-17T06:09:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.226+0000 7fbcd4f82640 1 -- 172.21.15.135:0/2713491773 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc001670 con 0x7fbcd00ba560 2024-04-17T06:09:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.227+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2713491773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd00ba560 msgr2=0x7fbcd00d42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.227+0000 7fbcd81e8640 1 --2- 172.21.15.135:0/2713491773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd00ba560 0x7fbcd00d42d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00097a0 tx=0x7fbcc002b820 comp rx=0 tx=0).stop 2024-04-17T06:09:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.227+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2713491773 shutdown_connections 2024-04-17T06:09:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.227+0000 7fbcd81e8640 1 --2- 172.21.15.135:0/2713491773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd00ba560 0x7fbcd00d42d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.227+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2713491773 >> 172.21.15.135:0/2713491773 conn(0x7fbcd006d270 msgr2=0x7fbcd006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.227+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2713491773 shutdown_connections 2024-04-17T06:09:17.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.227+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2713491773 wait complete. 2024-04-17T06:09:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.228+0000 7fbcd81e8640 1 Processor -- start 2024-04-17T06:09:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.228+0000 7fbcd81e8640 1 -- start start 2024-04-17T06:09:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd81e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd00ba560 0x7fbcd01613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd81e8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcd00d1850 0x7fbcd00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd81e8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd015ebb0 0x7fbcd00ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd81e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00d1e20 con 0x7fbcd00ba560 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd81e8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd00d1ca0 con 0x7fbcd015ebb0 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd81e8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd00d1fa0 con 0x7fbcd00d1850 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd00ba560 0x7fbcd01613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd00ba560 0x7fbcd01613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53662/0 (socket says 172.21.15.135:53662) 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 -- 172.21.15.135:0/2361326685 learned_addr learned my addr 172.21.15.135:0/2361326685 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd6785640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd015ebb0 0x7fbcd00ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5783640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcd00d1850 0x7fbcd00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 -- 172.21.15.135:0/2361326685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcd00d1850 msgr2=0x7fbcd00cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcd00d1850 0x7fbcd00cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 -- 172.21.15.135:0/2361326685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd015ebb0 msgr2=0x7fbcd00ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd015ebb0 0x7fbcd00ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 -- 172.21.15.135:0/2361326685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd00cc0e0 con 0x7fbcd00ba560 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcd5f84640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd00ba560 0x7fbcd01613c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c520 tx=0x7fbccc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcc67fc640 1 -- 172.21.15.135:0/2361326685 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc017070 con 0x7fbcd00ba560 2024-04-17T06:09:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcc67fc640 1 -- 172.21.15.135:0/2361326685 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbccc005d00 con 0x7fbcd00ba560 2024-04-17T06:09:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.229+0000 7fbcc67fc640 1 -- 172.21.15.135:0/2361326685 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc0072b0 con 0x7fbcd00ba560 2024-04-17T06:09:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.230+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd00cc3c0 con 0x7fbcd00ba560 2024-04-17T06:09:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.230+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd00cc980 con 0x7fbcd00ba560 2024-04-17T06:09:17.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.231+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd0074940 con 0x7fbcd00ba560 2024-04-17T06:09:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.231+0000 7fbcc67fc640 1 -- 172.21.15.135:0/2361326685 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbccc01a070 con 0x7fbcd00ba560 2024-04-17T06:09:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.231+0000 7fbcc67fc640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbcb406c8f0 0x7fbcb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.232+0000 7fbcc67fc640 1 -- 172.21.15.135:0/2361326685 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbccc08ac80 con 0x7fbcd00ba560 2024-04-17T06:09:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.232+0000 7fbcd5783640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbcb406c8f0 0x7fbcb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.232+0000 7fbcd5783640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbcb406c8f0 0x7fbcb406edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0035000 tx=0x7fbcc0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.236+0000 7fbcc67fc640 1 -- 172.21.15.135:0/2361326685 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbccc059400 con 0x7fbcd00ba560 2024-04-17T06:09:17.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.410+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbcd00b4ab0 con 0x7fbcd00ba560 2024-04-17T06:09:17.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.411+0000 7fbcc67fc640 1 -- 172.21.15.135:0/2361326685 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fbccc058d00 con 0x7fbcd00ba560 2024-04-17T06:09:17.409 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:17.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbcb406c8f0 msgr2=0x7fbcb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbcb406c8f0 0x7fbcb406edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0035000 tx=0x7fbcc0008030 comp rx=0 tx=0).stop 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd00ba560 msgr2=0x7fbcd01613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd00ba560 0x7fbcd01613c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c520 tx=0x7fbccc00c9e0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 shutdown_connections 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbcd015ebb0 0x7fbcd00ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbcd00d1850 0x7fbcd00cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbcb406c8f0 0x7fbcb406edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 --2- 172.21.15.135:0/2361326685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd00ba560 0x7fbcd01613c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 >> 172.21.15.135:0/2361326685 conn(0x7fbcd006d270 msgr2=0x7fbcd00b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 shutdown_connections 2024-04-17T06:09:17.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.413+0000 7fbcd81e8640 1 -- 172.21.15.135:0/2361326685 wait complete. 2024-04-17T06:09:17.422 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T06:09:17.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.528+0000 7f49dae57640 1 Processor -- start 2024-04-17T06:09:17.526 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.528+0000 7f49dae57640 1 -- start start 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49dae57640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49d415ebb0 0x7f49d40d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49dae57640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d40d3230 con 0x7f49d40ba560 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49dae57640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d40d3510 con 0x7f49d40d1850 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49dae57640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f49d40d37f0 con 0x7f49d415ebb0 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49d8bf3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49d415ebb0 0x7f49d40d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49cbfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f49d40ba560 0x7f49d40d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:49022/0 (socket says 172.21.15.135:49022) 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49cbfff640 1 -- 172.21.15.135:0/3365209862 learned_addr learned my addr 172.21.15.135:0/3365209862 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49d8bf3640 1 -- 172.21.15.135:0/3365209862 >> v1:172.21.15.190:6789/0 conn(0x7f49d40d1850 legacy=0x7f49d40b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49d8bf3640 1 -- 172.21.15.135:0/3365209862 >> v1:172.21.15.135:6789/0 conn(0x7f49d40ba560 legacy=0x7f49d40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49d8bf3640 1 -- 172.21.15.135:0/3365209862 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d40d3ad0 con 0x7f49d415ebb0 2024-04-17T06:09:17.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.529+0000 7f49cb7fe640 1 -- 172.21.15.135:0/3365209862 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539625065 0 0) 0x7f49d40d3230 con 0x7f49d40ba560 2024-04-17T06:09:17.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.530+0000 7f49d8bf3640 1 --2- 172.21.15.135:0/3365209862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d415ebb0 0x7f49d40d42d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49d00097a0 tx=0x7f49d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfa0f394016db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.530+0000 7f49cb7fe640 1 -- 172.21.15.135:0/3365209862 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0035020 con 0x7f49d415ebb0 2024-04-17T06:09:17.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.530+0000 7f49cb7fe640 1 -- 172.21.15.135:0/3365209862 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d0031830 con 0x7f49d415ebb0 2024-04-17T06:09:17.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.530+0000 7f49cb7fe640 1 -- 172.21.15.135:0/3365209862 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0031c00 con 0x7f49d415ebb0 2024-04-17T06:09:17.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.530+0000 7f49dae57640 1 -- 172.21.15.135:0/3365209862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d415ebb0 msgr2=0x7f49d40d42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.531+0000 7f49dae57640 1 --2- 172.21.15.135:0/3365209862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d415ebb0 0x7f49d40d42d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49d00097a0 tx=0x7f49d002b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.531+0000 7f49dae57640 1 -- 172.21.15.135:0/3365209862 shutdown_connections 2024-04-17T06:09:17.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.531+0000 7f49dae57640 1 --2- 172.21.15.135:0/3365209862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d415ebb0 0x7f49d40d42d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.531+0000 7f49dae57640 1 -- 172.21.15.135:0/3365209862 >> 172.21.15.135:0/3365209862 conn(0x7f49d406d270 msgr2=0x7f49d406d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:17.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.531+0000 7f49dae57640 1 -- 172.21.15.135:0/3365209862 shutdown_connections 2024-04-17T06:09:17.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.531+0000 7f49dae57640 1 -- 172.21.15.135:0/3365209862 wait complete. 2024-04-17T06:09:17.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.532+0000 7f49dae57640 1 Processor -- start 2024-04-17T06:09:17.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.532+0000 7f49dae57640 1 -- start start 2024-04-17T06:09:17.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49dae57640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d40ba560 0x7f49d414e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49dae57640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d40d1850 0x7f49d414ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49dae57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d415ebb0 0x7f49d40cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49dae57640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40d1e20 con 0x7f49d40d1850 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49dae57640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40d1ca0 con 0x7f49d40ba560 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49dae57640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f49d40d1fa0 con 0x7f49d415ebb0 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d40d1850 0x7f49d414ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d40d1850 0x7f49d414ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53684/0 (socket says 172.21.15.135:53684) 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 -- 172.21.15.135:0/1914751203 learned_addr learned my addr 172.21.15.135:0/1914751203 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49d8bf3640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d40ba560 0x7f49d414e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49d93f4640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d415ebb0 0x7f49d40cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 -- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d415ebb0 msgr2=0x7f49d40cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d415ebb0 0x7f49d40cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 -- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d40ba560 msgr2=0x7f49d414e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d40ba560 0x7f49d414e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 -- 172.21.15.135:0/1914751203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d414b740 con 0x7f49d40d1850 2024-04-17T06:09:17.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49d8bf3640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d40ba560 0x7f49d414e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49cbfff640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d40d1850 0x7f49d414ada0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c4d0 tx=0x7f49c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49c97fa640 1 -- 172.21.15.135:0/1914751203 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4018070 con 0x7f49d40d1850 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49c97fa640 1 -- 172.21.15.135:0/1914751203 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c4007500 con 0x7f49d40d1850 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d414ba20 con 0x7f49d40d1850 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49c97fa640 1 -- 172.21.15.135:0/1914751203 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4004a20 con 0x7f49d40d1850 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.533+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d414bfe0 con 0x7f49d40d1850 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.534+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d414c1e0 con 0x7f49d40d1850 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.535+0000 7f49c97fa640 1 -- 172.21.15.135:0/1914751203 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49c401b070 con 0x7f49d40d1850 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.535+0000 7f49c97fa640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c006c8f0 0x7f49c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.535+0000 7f49c97fa640 1 -- 172.21.15.135:0/1914751203 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f49c408aab0 con 0x7f49d40d1850 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.535+0000 7f49d8bf3640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c006c8f0 0x7f49c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.536+0000 7f49d8bf3640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c006c8f0 0x7f49c006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49d0036000 tx=0x7f49d003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.540+0000 7f49c97fa640 1 -- 172.21.15.135:0/1914751203 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49d414c1e0 con 0x7f49d40d1850 2024-04-17T06:09:17.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.720+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f49d4074940 con 0x7f49d40d1850 2024-04-17T06:09:17.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.720+0000 7f49c97fa640 1 -- 172.21.15.135:0/1914751203 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f49c4058be0 con 0x7f49d40d1850 2024-04-17T06:09:17.718 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:17.718 INFO:teuthology.orchestra.run.smithi135.stdout:[ 2024-04-17T06:09:17.718 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:09:17.718 INFO:teuthology.orchestra.run.smithi135.stdout: "poolnum": 1, 2024-04-17T06:09:17.719 INFO:teuthology.orchestra.run.smithi135.stdout: "poolname": ".mgr" 2024-04-17T06:09:17.719 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:09:17.719 INFO:teuthology.orchestra.run.smithi135.stdout:] 2024-04-17T06:09:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c006c8f0 msgr2=0x7f49c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c006c8f0 0x7f49c006edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49d0036000 tx=0x7f49d003c040 comp rx=0 tx=0).stop 2024-04-17T06:09:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d40d1850 msgr2=0x7f49d414ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d40d1850 0x7f49d414ada0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c4d0 tx=0x7f49c400c990 comp rx=0 tx=0).stop 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 shutdown_connections 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49d415ebb0 0x7f49d40cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c006c8f0 0x7f49c006edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d40d1850 0x7f49d414ada0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 --2- 172.21.15.135:0/1914751203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49d40ba560 0x7f49d414e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 >> 172.21.15.135:0/1914751203 conn(0x7f49d406d270 msgr2=0x7f49d40cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 shutdown_connections 2024-04-17T06:09:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.722+0000 7f49dae57640 1 -- 172.21.15.135:0/1914751203 wait complete. 2024-04-17T06:09:17.731 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T06:09:17.731 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T06:09:17.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.840+0000 7ff496d24640 1 Processor -- start 2024-04-17T06:09:17.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.840+0000 7ff496d24640 1 -- start start 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff496d24640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900b5f10 0x7ff4901449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff496d24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff49006ff90 0x7ff490144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff496d24640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4900b3c10 con 0x7ff49006fbb0 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff496d24640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff4900b3ef0 con 0x7ff49006ff90 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff496d24640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900b41d0 con 0x7ff4900b5f10 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff4952c1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff49006ff90 0x7ff490144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff4952c1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff49006ff90 0x7ff490144ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53700/0 (socket says 172.21.15.135:53700) 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff4952c1640 1 -- 172.21.15.135:0/1663180821 learned_addr learned my addr 172.21.15.135:0/1663180821 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff487fff640 1 --2- 172.21.15.135:0/1663180821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900b5f10 0x7ff4901449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff4952c1640 1 -- 172.21.15.135:0/1663180821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900b5f10 msgr2=0x7ff4901449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff4952c1640 1 --2- 172.21.15.135:0/1663180821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900b5f10 0x7ff4901449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff4952c1640 1 -- 172.21.15.135:0/1663180821 >> v1:172.21.15.190:6789/0 conn(0x7ff49006fbb0 legacy=0x7ff4900cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:17.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.841+0000 7ff4952c1640 1 -- 172.21.15.135:0/1663180821 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900b44b0 con 0x7ff49006ff90 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.842+0000 7ff4952c1640 1 --2- 172.21.15.135:0/1663180821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006ff90 0x7ff490144ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff488009e50 tx=0x7ff48802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb96dcca4c9b35e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.842+0000 7ff4877fe640 1 -- 172.21.15.135:0/1663180821 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48802ca70 con 0x7ff49006ff90 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.842+0000 7ff4877fe640 1 -- 172.21.15.135:0/1663180821 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48802cc10 con 0x7ff49006ff90 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.842+0000 7ff4877fe640 1 -- 172.21.15.135:0/1663180821 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4880343f0 con 0x7ff49006ff90 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.842+0000 7ff496d24640 1 -- 172.21.15.135:0/1663180821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006ff90 msgr2=0x7ff490144ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.842+0000 7ff496d24640 1 --2- 172.21.15.135:0/1663180821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006ff90 0x7ff490144ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff488009e50 tx=0x7ff48802bed0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.843+0000 7ff496d24640 1 -- 172.21.15.135:0/1663180821 shutdown_connections 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.843+0000 7ff496d24640 1 --2- 172.21.15.135:0/1663180821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006ff90 0x7ff490144ee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.843+0000 7ff496d24640 1 --2- 172.21.15.135:0/1663180821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4900b5f10 0x7ff4901449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.843+0000 7ff496d24640 1 -- 172.21.15.135:0/1663180821 >> 172.21.15.135:0/1663180821 conn(0x7ff4900b2350 msgr2=0x7ff4900b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:17.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.843+0000 7ff496d24640 1 -- 172.21.15.135:0/1663180821 shutdown_connections 2024-04-17T06:09:17.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.843+0000 7ff496d24640 1 -- 172.21.15.135:0/1663180821 wait complete. 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.844+0000 7ff496d24640 1 Processor -- start 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.844+0000 7ff496d24640 1 -- start start 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff496d24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006fbb0 0x7ff490136880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff496d24640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49006ff90 0x7ff49013fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff496d24640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900b5f10 0x7ff49006e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff496d24640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900b3860 con 0x7ff49006fbb0 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff496d24640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff490165280 con 0x7ff4900b5f10 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff496d24640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff490135890 con 0x7ff49006ff90 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006fbb0 0x7ff490136880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006fbb0 0x7ff490136880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53702/0 (socket says 172.21.15.135:53702) 2024-04-17T06:09:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 -- 172.21.15.135:0/3136687444 learned_addr learned my addr 172.21.15.135:0/3136687444 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff487fff640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49006ff90 0x7ff49013fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff4952c1640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900b5f10 0x7ff49006e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 -- 172.21.15.135:0/3136687444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49006ff90 msgr2=0x7ff49013fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49006ff90 0x7ff49013fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 -- 172.21.15.135:0/3136687444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900b5f10 msgr2=0x7ff49006e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900b5f10 0x7ff49006e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 -- 172.21.15.135:0/3136687444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901349a0 con 0x7ff49006fbb0 2024-04-17T06:09:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff494ac0640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006fbb0 0x7ff490136880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00c430 tx=0x7ff48c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff4857fa640 1 -- 172.21.15.135:0/3136687444 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c011790 con 0x7ff49006fbb0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff4857fa640 1 -- 172.21.15.135:0/3136687444 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c017070 con 0x7ff49006fbb0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff490134c80 con 0x7ff49006fbb0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff4857fa640 1 -- 172.21.15.135:0/3136687444 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c011bb0 con 0x7ff49006fbb0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.845+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff490135240 con 0x7ff49006fbb0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.846+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4900749b0 con 0x7ff49006fbb0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.847+0000 7ff4857fa640 1 -- 172.21.15.135:0/3136687444 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff48c01c460 con 0x7ff49006fbb0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.847+0000 7ff4857fa640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47c06c8f0 0x7ff47c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.847+0000 7ff487fff640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47c06c8f0 0x7ff47c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.847+0000 7ff4857fa640 1 -- 172.21.15.135:0/3136687444 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff48c08b520 con 0x7ff49006fbb0 2024-04-17T06:09:17.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.848+0000 7ff487fff640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47c06c8f0 0x7ff47c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff490137890 tx=0x7ff480008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:17.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:17.852+0000 7ff4857fa640 1 -- 172.21.15.135:0/3136687444 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff48c059650 con 0x7ff49006fbb0 2024-04-17T06:09:18.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.028+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff49015efc0 con 0x7ff49006fbb0 2024-04-17T06:09:18.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.602+0000 7ff4857fa640 1 -- 172.21.15.135:0/3136687444 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7ff48c05d950 con 0x7ff49006fbb0 2024-04-17T06:09:18.601 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'cephfs_metadata' created 2024-04-17T06:09:18.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47c06c8f0 msgr2=0x7ff47c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:18.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47c06c8f0 0x7ff47c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff490137890 tx=0x7ff480008040 comp rx=0 tx=0).stop 2024-04-17T06:09:18.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006fbb0 msgr2=0x7ff490136880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:18.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006fbb0 0x7ff490136880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00c430 tx=0x7ff48c00c8f0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 shutdown_connections 2024-04-17T06:09:18.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900b5f10 0x7ff49006e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49006ff90 0x7ff49013fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47c06c8f0 0x7ff47c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 --2- 172.21.15.135:0/3136687444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49006fbb0 0x7ff490136880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 >> 172.21.15.135:0/3136687444 conn(0x7ff4900b2350 msgr2=0x7ff4900b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:18.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 shutdown_connections 2024-04-17T06:09:18.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.605+0000 7ff496d24640 1 -- 172.21.15.135:0/3136687444 wait complete. 2024-04-17T06:09:18.619 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-17T06:09:18.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.733+0000 7f7952559640 1 Processor -- start 2024-04-17T06:09:18.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.733+0000 7f7952559640 1 -- start start 2024-04-17T06:09:18.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.733+0000 7f7952559640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f794c0cd060 0x7f794c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:18.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.733+0000 7f7952559640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f794c0cc2e0 con 0x7f794c0b4b90 2024-04-17T06:09:18.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.733+0000 7f7952559640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f794c0cc580 con 0x7f794c15b8b0 2024-04-17T06:09:18.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.733+0000 7f7952559640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f794c0cd4c0 con 0x7f794c0cd060 2024-04-17T06:09:18.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.734+0000 7f7943fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f794c0cd060 0x7f794c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:18.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.734+0000 7f7950af6640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f794c0b4b90 0x7f794c0cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:49036/0 (socket says 172.21.15.135:49036) 2024-04-17T06:09:18.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.734+0000 7f7943fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f794c0cd060 0x7f794c0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53708/0 (socket says 172.21.15.135:53708) 2024-04-17T06:09:18.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.734+0000 7f7950af6640 1 -- 172.21.15.135:0/2004327561 learned_addr learned my addr 172.21.15.135:0/2004327561 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:18.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.734+0000 7f7943fff640 1 -- 172.21.15.135:0/2004327561 >> v1:172.21.15.190:6789/0 conn(0x7f794c15b8b0 legacy=0x7f794c0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:18.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.734+0000 7f7943fff640 1 -- 172.21.15.135:0/2004327561 >> v1:172.21.15.135:6789/0 conn(0x7f794c0b4b90 legacy=0x7f794c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:18.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.734+0000 7f7943fff640 1 -- 172.21.15.135:0/2004327561 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c0cc860 con 0x7f794c0cd060 2024-04-17T06:09:18.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f7943fff640 1 --2- 172.21.15.135:0/2004327561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c0ce7c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f79340097a0 tx=0x7f793402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba65dd7f01ccd019 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:18.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f794b7fe640 1 -- 172.21.15.135:0/2004327561 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934035020 con 0x7f794c0cd060 2024-04-17T06:09:18.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f794b7fe640 1 -- 172.21.15.135:0/2004327561 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7934031830 con 0x7f794c0cd060 2024-04-17T06:09:18.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f794b7fe640 1 -- 172.21.15.135:0/2004327561 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934031c00 con 0x7f794c0cd060 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f7952559640 1 -- 172.21.15.135:0/2004327561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 msgr2=0x7f794c0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f7952559640 1 --2- 172.21.15.135:0/2004327561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c0ce7c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f79340097a0 tx=0x7f793402b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f7952559640 1 -- 172.21.15.135:0/2004327561 shutdown_connections 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f7952559640 1 --2- 172.21.15.135:0/2004327561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c0ce7c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f7952559640 1 -- 172.21.15.135:0/2004327561 >> 172.21.15.135:0/2004327561 conn(0x7f794c09a9f0 msgr2=0x7f794c0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f7952559640 1 -- 172.21.15.135:0/2004327561 shutdown_connections 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.735+0000 7f7952559640 1 -- 172.21.15.135:0/2004327561 wait complete. 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.736+0000 7f7952559640 1 Processor -- start 2024-04-17T06:09:18.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7952559640 1 -- start start 2024-04-17T06:09:18.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7952559640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f794c0b4b90 0x7f794c15d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:18.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7952559640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7952559640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c15b8b0 0x7f794c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7952559640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f794c15bc90 con 0x7f794c0cd060 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7952559640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0ced00 con 0x7f794c0b4b90 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7952559640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f794c0cef60 con 0x7f794c15b8b0 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7943fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7943fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53724/0 (socket says 172.21.15.135:53724) 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.737+0000 7f7943fff640 1 -- 172.21.15.135:0/4252357317 learned_addr learned my addr 172.21.15.135:0/4252357317 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7950af6640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c15b8b0 0x7f794c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7943fff640 1 -- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c15b8b0 msgr2=0x7f794c0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:18.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7943fff640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c15b8b0 0x7f794c0c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f794bfff640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f794c0b4b90 0x7f794c15d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7943fff640 1 -- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f794c0b4b90 msgr2=0x7f794c15d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7943fff640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f794c0b4b90 0x7f794c15d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7943fff640 1 -- 172.21.15.135:0/4252357317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c0c8c40 con 0x7f794c0cd060 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7950af6640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c15b8b0 0x7f794c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f794bfff640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f794c0b4b90 0x7f794c15d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7943fff640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c157dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79340098c0 tx=0x7f79340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f79497fa640 1 -- 172.21.15.135:0/4252357317 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934041070 con 0x7f794c0cd060 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f79497fa640 1 -- 172.21.15.135:0/4252357317 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793403aa30 con 0x7f794c0cd060 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794c0c8ee0 con 0x7f794c0cd060 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f79497fa640 1 -- 172.21.15.135:0/4252357317 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934035040 con 0x7f794c0cd060 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.738+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794c0c94a0 con 0x7f794c0cd060 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.739+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794c068a20 con 0x7f794c0cd060 2024-04-17T06:09:18.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.740+0000 7f79497fa640 1 -- 172.21.15.135:0/4252357317 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f793404c060 con 0x7f794c0cd060 2024-04-17T06:09:18.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.740+0000 7f79497fa640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f792c06c920 0x7f792c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:18.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.740+0000 7f79497fa640 1 -- 172.21.15.135:0/4252357317 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f79340ad360 con 0x7f794c0cd060 2024-04-17T06:09:18.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.740+0000 7f794bfff640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f792c06c920 0x7f792c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:18.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.740+0000 7f794bfff640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f792c06c920 0x7f792c06ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f793c003eb0 tx=0x7f793c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:18.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.745+0000 7f79497fa640 1 -- 172.21.15.135:0/4252357317 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f793407ba00 con 0x7f794c0cd060 2024-04-17T06:09:18.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:18.919+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f794c0b75b0 con 0x7f794c0cd060 2024-04-17T06:09:19.385 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:09:19.388 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:09:19.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.605+0000 7f79497fa640 1 -- 172.21.15.135:0/4252357317 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f793407b300 con 0x7f794c0cd060 2024-04-17T06:09:19.603 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'cephfs_data' created 2024-04-17T06:09:19.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f792c06c920 msgr2=0x7f792c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:19.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f792c06c920 0x7f792c06ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f793c003eb0 tx=0x7f793c009040 comp rx=0 tx=0).stop 2024-04-17T06:09:19.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 msgr2=0x7f794c157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:19.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c157dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79340098c0 tx=0x7f79340046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 shutdown_connections 2024-04-17T06:09:19.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f794c15b8b0 0x7f794c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f792c06c920 0x7f792c06ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f794c0cd060 0x7f794c157dc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 --2- 172.21.15.135:0/4252357317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f794c0b4b90 0x7f794c15d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.607+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 >> 172.21.15.135:0/4252357317 conn(0x7f794c09a9f0 msgr2=0x7f794c0b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:19.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.608+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 shutdown_connections 2024-04-17T06:09:19.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.608+0000 7f7952559640 1 -- 172.21.15.135:0/4252357317 wait complete. 2024-04-17T06:09:19.628 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T06:09:19.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.740+0000 7fd88c359640 1 Processor -- start 2024-04-17T06:09:19.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.740+0000 7fd88c359640 1 -- start start 2024-04-17T06:09:19.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88c359640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd88415c930 0x7fd88409b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:19.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88c359640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd88409b100 0x7fd8840b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:19.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88c359640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8840cd440 con 0x7fd8840d0490 2024-04-17T06:09:19.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88c359640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd8840c7210 con 0x7fd88409b100 2024-04-17T06:09:19.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88c359640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd8840cd720 con 0x7fd88415c930 2024-04-17T06:09:19.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88a8f6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd88409b100 0x7fd8840b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:19.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88a8f6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd88409b100 0x7fd8840b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53732/0 (socket says 172.21.15.135:53732) 2024-04-17T06:09:19.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88a8f6640 1 -- 172.21.15.135:0/6846355 learned_addr learned my addr 172.21.15.135:0/6846355 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:19.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.741+0000 7fd88a0f5640 1 --2- 172.21.15.135:0/6846355 >> v2:172.21.15.190:3300/0 conn(0x7fd88415c930 0x7fd88409b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:19.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.742+0000 7fd88a8f6640 1 -- 172.21.15.135:0/6846355 >> v2:172.21.15.190:3300/0 conn(0x7fd88415c930 msgr2=0x7fd88409b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:19.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.742+0000 7fd88a8f6640 1 --2- 172.21.15.135:0/6846355 >> v2:172.21.15.190:3300/0 conn(0x7fd88415c930 0x7fd88409b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.742+0000 7fd88a8f6640 1 -- 172.21.15.135:0/6846355 >> v1:172.21.15.190:6789/0 conn(0x7fd8840d0490 legacy=0x7fd88409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.742+0000 7fd8890f3640 1 -- 172.21.15.135:0/6846355 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591207985 0 0) 0x7fd8840cd440 con 0x7fd8840d0490 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.742+0000 7fd88a8f6640 1 -- 172.21.15.135:0/6846355 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8840cda00 con 0x7fd88409b100 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.742+0000 7fd88a8f6640 1 --2- 172.21.15.135:0/6846355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88409b100 0x7fd8840b3300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd88406c570 tx=0x7fd87c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d58f60f81a27554 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd8890f3640 1 -- 172.21.15.135:0/6846355 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c035020 con 0x7fd88409b100 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd8890f3640 1 -- 172.21.15.135:0/6846355 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87c031870 con 0x7fd88409b100 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd8890f3640 1 -- 172.21.15.135:0/6846355 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c031c40 con 0x7fd88409b100 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd88c359640 1 -- 172.21.15.135:0/6846355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88409b100 msgr2=0x7fd8840b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:19.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd88c359640 1 --2- 172.21.15.135:0/6846355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88409b100 0x7fd8840b3300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd88406c570 tx=0x7fd87c02b910 comp rx=0 tx=0).stop 2024-04-17T06:09:19.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd88c359640 1 -- 172.21.15.135:0/6846355 shutdown_connections 2024-04-17T06:09:19.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd88c359640 1 --2- 172.21.15.135:0/6846355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88409b100 0x7fd8840b3300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd88c359640 1 --2- 172.21.15.135:0/6846355 >> v2:172.21.15.190:3300/0 conn(0x7fd88415c930 0x7fd88409b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd88c359640 1 -- 172.21.15.135:0/6846355 >> 172.21.15.135:0/6846355 conn(0x7fd8840b41a0 msgr2=0x7fd8840b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:19.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd88c359640 1 -- 172.21.15.135:0/6846355 shutdown_connections 2024-04-17T06:09:19.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.743+0000 7fd88c359640 1 -- 172.21.15.135:0/6846355 wait complete. 2024-04-17T06:09:19.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.744+0000 7fd88c359640 1 Processor -- start 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88c359640 1 -- start start 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88c359640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd88409b100 0x7fd88414f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88c359640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8840d0490 0x7fd88409c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88c359640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88415c930 0x7fd88409c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88c359640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd88409be10 con 0x7fd88415c930 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88c359640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd8840cdfc0 con 0x7fd88409b100 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88c359640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8840ce4e0 con 0x7fd8840d0490 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88a8f6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88415c930 0x7fd88409c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88a8f6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88415c930 0x7fd88409c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53748/0 (socket says 172.21.15.135:53748) 2024-04-17T06:09:19.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88a8f6640 1 -- 172.21.15.135:0/3119794213 learned_addr learned my addr 172.21.15.135:0/3119794213 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd88a0f5640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd88409b100 0x7fd88414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.745+0000 7fd8898f4640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8840d0490 0x7fd88409c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd88a8f6640 1 -- 172.21.15.135:0/3119794213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8840d0490 msgr2=0x7fd88409c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd88a8f6640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8840d0490 0x7fd88409c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd88a8f6640 1 -- 172.21.15.135:0/3119794213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd88409b100 msgr2=0x7fd88414f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd88a8f6640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd88409b100 0x7fd88414f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd88a8f6640 1 -- 172.21.15.135:0/3119794213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88409e240 con 0x7fd88415c930 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd88a8f6640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88415c930 0x7fd88409c980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009960 tx=0x7fd87c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd882ffd640 1 -- 172.21.15.135:0/3119794213 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c035040 con 0x7fd88415c930 2024-04-17T06:09:19.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd882ffd640 1 -- 172.21.15.135:0/3119794213 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87c030d00 con 0x7fd88415c930 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd882ffd640 1 -- 172.21.15.135:0/3119794213 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c0456a0 con 0x7fd88415c930 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88409e500 con 0x7fd88415c930 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.746+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88409eac0 con 0x7fd88415c930 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.747+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd884068a20 con 0x7fd88415c930 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.748+0000 7fd882ffd640 1 -- 172.21.15.135:0/3119794213 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd87c038020 con 0x7fd88415c930 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.748+0000 7fd882ffd640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd87006c8f0 0x7fd87006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.748+0000 7fd88a0f5640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd87006c8f0 0x7fd87006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.748+0000 7fd882ffd640 1 -- 172.21.15.135:0/3119794213 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fd87c0adc30 con 0x7fd88415c930 2024-04-17T06:09:19.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.748+0000 7fd88a0f5640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd87006c8f0 0x7fd87006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8840b4d30 tx=0x7fd87400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:19.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.753+0000 7fd882ffd640 1 -- 172.21.15.135:0/3119794213 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd87c07bb00 con 0x7fd88415c930 2024-04-17T06:09:19.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:19.966+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd8840b5040 con 0x7fd88415c930 2024-04-17T06:09:20.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.634+0000 7fd882ffd640 1 -- 172.21.15.135:0/3119794213 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fd87c07fe00 con 0x7fd88415c930 2024-04-17T06:09:20.634 INFO:teuthology.orchestra.run.smithi135.stderr:new fs with metadata pool 2 and data pool 3 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.643+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd87006c8f0 msgr2=0x7fd87006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.643+0000 7fd88c359640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd87006c8f0 0x7fd87006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8840b4d30 tx=0x7fd87400b040 comp rx=0 tx=0).stop 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.643+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88415c930 msgr2=0x7fd88409c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.643+0000 7fd88c359640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88415c930 0x7fd88409c980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009960 tx=0x7fd87c0046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.644+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 shutdown_connections 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.644+0000 7fd88c359640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd87006c8f0 0x7fd87006edb0 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8840b4d30 tx=0x7fd87400b040 comp rx=0 tx=0).stop 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.644+0000 7fd88c359640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd88415c930 0x7fd88409c980 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.644+0000 7fd88c359640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8840d0490 0x7fd88409c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.644+0000 7fd88c359640 1 --2- 172.21.15.135:0/3119794213 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd88409b100 0x7fd88414f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.644+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 >> 172.21.15.135:0/3119794213 conn(0x7fd8840b41a0 msgr2=0x7fd884074e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:20.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.647+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 shutdown_connections 2024-04-17T06:09:20.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.647+0000 7fd88c359640 1 -- 172.21.15.135:0/3119794213 wait complete. 2024-04-17T06:09:20.659 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-17T06:09:20.659 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-17T06:09:20.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.771+0000 7fc43d269640 1 Processor -- start 2024-04-17T06:09:20.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.771+0000 7fc43d269640 1 -- start start 2024-04-17T06:09:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc43d269640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4380d1a60 0x7fc438169760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:20.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc43d269640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4380d3100 con 0x7fc4380d4470 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc43d269640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4380d33d0 con 0x7fc43815eca0 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc43d269640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc43815f080 con 0x7fc4380d1a60 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc42e7fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc4380d4470 0x7fc4380d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:49042/0 (socket says 172.21.15.135:49042) 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc42e7fc640 1 -- 172.21.15.135:0/2236161772 learned_addr learned my addr 172.21.15.135:0/2236161772 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc4377fe640 1 --2- 172.21.15.135:0/2236161772 >> v2:172.21.15.190:3300/0 conn(0x7fc4380d1a60 0x7fc438169760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4260169689 0 0) 0x7fc4380d3100 con 0x7fc4380d4470 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.772+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc424003520 con 0x7fc4380d4470 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3471943369 0 0) 0x7fc424003520 con 0x7fc4380d4470 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 >> v2:172.21.15.190:3300/0 conn(0x7fc4380d1a60 msgr2=0x7fc438169760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 --2- 172.21.15.135:0/2236161772 >> v2:172.21.15.190:3300/0 conn(0x7fc4380d1a60 0x7fc438169760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 >> v1:172.21.15.190:6789/0 conn(0x7fc43815eca0 legacy=0x7fc4380d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:20.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4380d36b0 con 0x7fc4380d4470 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4377fe640 1 --2- 172.21.15.135:0/2236161772 >> v2:172.21.15.190:3300/0 conn(0x7fc4380d1a60 0x7fc438169760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fc420001670 con 0x7fc4380d4470 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 >> v1:172.21.15.135:6789/0 conn(0x7fc4380d4470 legacy=0x7fc4380d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240039b0 0x7fc424003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424004800 0x7fc424004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240051c0 0x7fc42400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc43815f080 con 0x7fc4240039b0 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc42400c070 con 0x7fc4240051c0 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc42400c270 con 0x7fc424004800 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc436ffd640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240039b0 0x7fc424003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:20.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4377fe640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240051c0 0x7fc42400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc42e7fc640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424004800 0x7fc424004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc436ffd640 1 -- 172.21.15.135:0/2236161772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424004800 msgr2=0x7fc424004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc436ffd640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc424004800 0x7fc424004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc436ffd640 1 -- 172.21.15.135:0/2236161772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240051c0 msgr2=0x7fc42400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc436ffd640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4240051c0 0x7fc42400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc4377fe640 1 -- 172.21.15.135:0/2236161772 reap_dead start 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.773+0000 7fc436ffd640 1 -- 172.21.15.135:0/2236161772 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc424003520 con 0x7fc4240039b0 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.774+0000 7fc436ffd640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240039b0 0x7fc424003dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc428001750 tx=0x7fc424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34388ad407c967 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.774+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428001370 con 0x7fc4240039b0 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.774+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4280052d0 con 0x7fc4240039b0 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.774+0000 7fc4367fc640 1 -- 172.21.15.135:0/2236161772 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428003830 con 0x7fc4240039b0 2024-04-17T06:09:20.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.775+0000 7fc43d269640 1 -- 172.21.15.135:0/2236161772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240039b0 msgr2=0x7fc424003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:20.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.775+0000 7fc43d269640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240039b0 0x7fc424003dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc428001750 tx=0x7fc424005830 comp rx=0 tx=0).stop 2024-04-17T06:09:20.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.775+0000 7fc43d269640 1 -- 172.21.15.135:0/2236161772 shutdown_connections 2024-04-17T06:09:20.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.775+0000 7fc43d269640 1 --2- 172.21.15.135:0/2236161772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4240039b0 0x7fc424003dd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.775+0000 7fc43d269640 1 -- 172.21.15.135:0/2236161772 >> 172.21.15.135:0/2236161772 conn(0x7fc4380b64e0 msgr2=0x7fc4380b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:20.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.775+0000 7fc43d269640 1 -- 172.21.15.135:0/2236161772 shutdown_connections 2024-04-17T06:09:20.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.775+0000 7fc43d269640 1 -- 172.21.15.135:0/2236161772 wait complete. 2024-04-17T06:09:20.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.776+0000 7fc43d269640 1 Processor -- start 2024-04-17T06:09:20.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.776+0000 7fc43d269640 1 -- start start 2024-04-17T06:09:20.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc43d269640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4240039b0 0x7fc438154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:20.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc43d269640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc438151b10 0x7fc43814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:20.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc43d269640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380ca6b0 0x7fc4380cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:20.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc43d269640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4380d3f50 con 0x7fc438151b10 2024-04-17T06:09:20.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc43d269640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc4380d3c70 con 0x7fc4380ca6b0 2024-04-17T06:09:20.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc43d269640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc4380ba490 con 0x7fc4240039b0 2024-04-17T06:09:20.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc438151b10 0x7fc43814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc438151b10 0x7fc43814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53766/0 (socket says 172.21.15.135:53766) 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 -- 172.21.15.135:0/2031026380 learned_addr learned my addr 172.21.15.135:0/2031026380 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc436ffd640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4240039b0 0x7fc438154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc4377fe640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380ca6b0 0x7fc4380cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 -- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4240039b0 msgr2=0x7fc438154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4240039b0 0x7fc438154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 -- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380ca6b0 msgr2=0x7fc4380cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380ca6b0 0x7fc4380cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 -- 172.21.15.135:0/2031026380 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4381f6220 con 0x7fc438151b10 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc4377fe640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380ca6b0 0x7fc4380cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42e7fc640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc438151b10 0x7fc43814dd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4200011e0 tx=0x7fc420007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42ffff640 1 -- 172.21.15.135:0/2031026380 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42000f960 con 0x7fc438151b10 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42ffff640 1 -- 172.21.15.135:0/2031026380 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc420018070 con 0x7fc438151b10 2024-04-17T06:09:20.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4381f6500 con 0x7fc438151b10 2024-04-17T06:09:20.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.777+0000 7fc42ffff640 1 -- 172.21.15.135:0/2031026380 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42000fd60 con 0x7fc438151b10 2024-04-17T06:09:20.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.778+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4381f6ac0 con 0x7fc438151b10 2024-04-17T06:09:20.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.778+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4380749b0 con 0x7fc438151b10 2024-04-17T06:09:20.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.779+0000 7fc42ffff640 1 -- 172.21.15.135:0/2031026380 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc42001d460 con 0x7fc438151b10 2024-04-17T06:09:20.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.779+0000 7fc42ffff640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc41806c8f0 0x7fc41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:20.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.779+0000 7fc42ffff640 1 -- 172.21.15.135:0/2031026380 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fc42008bbb0 con 0x7fc438151b10 2024-04-17T06:09:20.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.779+0000 7fc436ffd640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc41806c8f0 0x7fc41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:20.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.780+0000 7fc436ffd640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc41806c8f0 0x7fc41806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc428033000 tx=0x7fc428009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:20.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.784+0000 7fc42ffff640 1 -- 172.21.15.135:0/2031026380 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc420059fe0 con 0x7fc438151b10 2024-04-17T06:09:20.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:20.968+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fc4380b9ea0 con 0x7fc438151b10 2024-04-17T06:09:21.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.633+0000 7fc42ffff640 1 -- 172.21.15.135:0/2031026380 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7fc42005ee20 con 0x7fc438151b10 2024-04-17T06:09:21.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc41806c8f0 msgr2=0x7fc41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:21.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc41806c8f0 0x7fc41806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc428033000 tx=0x7fc428009030 comp rx=0 tx=0).stop 2024-04-17T06:09:21.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc438151b10 msgr2=0x7fc43814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc438151b10 0x7fc43814dd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4200011e0 tx=0x7fc420007720 comp rx=0 tx=0).stop 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 shutdown_connections 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4380ca6b0 0x7fc4380cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc41806c8f0 0x7fc41806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc438151b10 0x7fc43814dd50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 --2- 172.21.15.135:0/2031026380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4240039b0 0x7fc438154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 >> 172.21.15.135:0/2031026380 conn(0x7fc4380b64e0 msgr2=0x7fc4380d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 shutdown_connections 2024-04-17T06:09:21.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.637+0000 7fc43d269640 1 -- 172.21.15.135:0/2031026380 wait complete. 2024-04-17T06:09:21.649 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T06:09:21.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.767+0000 7f125647e640 1 Processor -- start 2024-04-17T06:09:21.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.767+0000 7f125647e640 1 -- start start 2024-04-17T06:09:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.768+0000 7f125647e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1250144980 0x7f12500cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:21.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.768+0000 7f125647e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f12500b2de0 0x7f125015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.768+0000 7f125647e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12500cff10 con 0x7f125015c290 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.768+0000 7f125647e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f12500d01f0 con 0x7f1250144980 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.768+0000 7f125647e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f12500b1890 con 0x7f12500b2de0 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.768+0000 7f124ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1250144980 0x7f12500cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.768+0000 7f124ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1250144980 0x7f12500cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53774/0 (socket says 172.21.15.135:53774) 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.769+0000 7f124ffff640 1 -- 172.21.15.135:0/2155344746 learned_addr learned my addr 172.21.15.135:0/2155344746 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.769+0000 7f1247fff640 1 --2- 172.21.15.135:0/2155344746 >> v2:172.21.15.190:3300/0 conn(0x7f12500b2de0 0x7f125015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.769+0000 7f124ffff640 1 -- 172.21.15.135:0/2155344746 >> v2:172.21.15.190:3300/0 conn(0x7f12500b2de0 msgr2=0x7f125015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:21.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.769+0000 7f124ffff640 1 --2- 172.21.15.135:0/2155344746 >> v2:172.21.15.190:3300/0 conn(0x7f12500b2de0 0x7f125015bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.769+0000 7f124ffff640 1 -- 172.21.15.135:0/2155344746 >> v1:172.21.15.190:6789/0 conn(0x7f125015c290 legacy=0x7f125006f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:21.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.769+0000 7f124ffff640 1 -- 172.21.15.135:0/2155344746 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12500b1b70 con 0x7f1250144980 2024-04-17T06:09:21.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.769+0000 7f124ffff640 1 --2- 172.21.15.135:0/2155344746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f12500cf8a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1250070440 tx=0x7f124002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f196a01de9a08e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:21.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f124f7fe640 1 -- 172.21.15.135:0/2155344746 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124003b070 con 0x7f1250144980 2024-04-17T06:09:21.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f124f7fe640 1 -- 172.21.15.135:0/2155344746 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240036440 con 0x7f1250144980 2024-04-17T06:09:21.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f124f7fe640 1 -- 172.21.15.135:0/2155344746 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240036810 con 0x7f1250144980 2024-04-17T06:09:21.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f125647e640 1 -- 172.21.15.135:0/2155344746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 msgr2=0x7f12500cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:21.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f125647e640 1 --2- 172.21.15.135:0/2155344746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f12500cf8a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1250070440 tx=0x7f124002e0e0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f125647e640 1 -- 172.21.15.135:0/2155344746 shutdown_connections 2024-04-17T06:09:21.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f125647e640 1 --2- 172.21.15.135:0/2155344746 >> v2:172.21.15.190:3300/0 conn(0x7f12500b2de0 0x7f125015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f125647e640 1 --2- 172.21.15.135:0/2155344746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f12500cf8a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f125647e640 1 -- 172.21.15.135:0/2155344746 >> 172.21.15.135:0/2155344746 conn(0x7f1250067d50 msgr2=0x7f125006b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:21.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f125647e640 1 -- 172.21.15.135:0/2155344746 shutdown_connections 2024-04-17T06:09:21.771 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.770+0000 7f125647e640 1 -- 172.21.15.135:0/2155344746 wait complete. 2024-04-17T06:09:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.771+0000 7f125647e640 1 Processor -- start 2024-04-17T06:09:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.771+0000 7f125647e640 1 -- start start 2024-04-17T06:09:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f125647e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12500b2de0 0x7f12501e3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f125647e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f125015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f125647e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125015c290 0x7f12500b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f125647e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12500b33c0 con 0x7f1250144980 2024-04-17T06:09:21.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f125647e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12500b3240 con 0x7f125015c290 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f125647e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f125006f7e0 con 0x7f12500b2de0 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f1247fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f125015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f1247fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f125015b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53778/0 (socket says 172.21.15.135:53778) 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f1247fff640 1 -- 172.21.15.135:0/2722757807 learned_addr learned my addr 172.21.15.135:0/2722757807 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f124ffff640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12500b2de0 0x7f12501e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.772+0000 7f1254a1b640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125015c290 0x7f12500b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f1247fff640 1 -- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12500b2de0 msgr2=0x7f12501e3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f1247fff640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12500b2de0 0x7f12501e3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f1247fff640 1 -- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125015c290 msgr2=0x7f12500b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f1247fff640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125015c290 0x7f12500b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f1247fff640 1 -- 172.21.15.135:0/2722757807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125015b5e0 con 0x7f1250144980 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f1254a1b640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125015c290 0x7f12500b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:21.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f1247fff640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f125015b0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f123800d830 tx=0x7f123800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f124d7fa640 1 -- 172.21.15.135:0/2722757807 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123800b6c0 con 0x7f1250144980 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f124d7fa640 1 -- 172.21.15.135:0/2722757807 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1238019070 con 0x7f1250144980 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f124d7fa640 1 -- 172.21.15.135:0/2722757807 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123800bac0 con 0x7f1250144980 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12501f0bf0 con 0x7f1250144980 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.773+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12501f11b0 con 0x7f1250144980 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.774+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12500749b0 con 0x7f1250144980 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.774+0000 7f124d7fa640 1 -- 172.21.15.135:0/2722757807 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f123801e460 con 0x7f1250144980 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.775+0000 7f124d7fa640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f123006c8f0 0x7f123006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.775+0000 7f124d7fa640 1 -- 172.21.15.135:0/2722757807 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f123808c310 con 0x7f1250144980 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.775+0000 7f124ffff640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f123006c8f0 0x7f123006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:21.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.775+0000 7f124ffff640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f123006c8f0 0x7f123006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1240038000 tx=0x7f1240032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:21.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.779+0000 7f124d7fa640 1 -- 172.21.15.135:0/2722757807 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f123805a6a0 con 0x7f1250144980 2024-04-17T06:09:21.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:21.954+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f125015cfd0 con 0x7f1250144980 2024-04-17T06:09:23.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.649+0000 7f124d7fa640 1 -- 172.21.15.135:0/2722757807 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f1238059fa0 con 0x7f1250144980 2024-04-17T06:09:23.648 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'cephfs_data_ec' created 2024-04-17T06:09:23.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.653+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f123006c8f0 msgr2=0x7f123006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:23.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.653+0000 7f125647e640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f123006c8f0 0x7f123006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1240038000 tx=0x7f1240032000 comp rx=0 tx=0).stop 2024-04-17T06:09:23.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.653+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 msgr2=0x7f125015b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:23.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.653+0000 7f125647e640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f125015b0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f123800d830 tx=0x7f123800dcf0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.654+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 shutdown_connections 2024-04-17T06:09:23.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.654+0000 7f125647e640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f125015c290 0x7f12500b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.654+0000 7f125647e640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f123006c8f0 0x7f123006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.654+0000 7f125647e640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1250144980 0x7f125015b0a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.654+0000 7f125647e640 1 --2- 172.21.15.135:0/2722757807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12500b2de0 0x7f12501e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.654+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 >> 172.21.15.135:0/2722757807 conn(0x7f1250067d50 msgr2=0x7f125015c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:23.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.654+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 shutdown_connections 2024-04-17T06:09:23.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.654+0000 7f125647e640 1 -- 172.21.15.135:0/2722757807 wait complete. 2024-04-17T06:09:23.671 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-17T06:09:23.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.785+0000 7f59cec14640 1 Processor -- start 2024-04-17T06:09:23.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.785+0000 7f59cec14640 1 -- start start 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cec14640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59c80cc840 0x7f59c8140710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cec14640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f59c80cccf0 0x7f59c8140c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cec14640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c813e3b0 con 0x7f59c813dfd0 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cec14640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f59c80ced80 con 0x7f59c80cccf0 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cec14640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f59c80cd290 con 0x7f59c80cc840 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cd1b1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f59c80cccf0 0x7f59c8140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cd1b1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f59c80cccf0 0x7f59c8140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53794/0 (socket says 172.21.15.135:53794) 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cd1b1640 1 -- 172.21.15.135:0/2554779057 learned_addr learned my addr 172.21.15.135:0/2554779057 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cc9b0640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f59c813dfd0 0x7f59c806e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:49050/0 (socket says 172.21.15.135:49050) 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cd1b1640 1 -- 172.21.15.135:0/2554779057 >> v2:172.21.15.190:3300/0 conn(0x7f59c80cc840 msgr2=0x7f59c8140710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cd1b1640 1 --2- 172.21.15.135:0/2554779057 >> v2:172.21.15.190:3300/0 conn(0x7f59c80cc840 0x7f59c8140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cd1b1640 1 -- 172.21.15.135:0/2554779057 >> v1:172.21.15.135:6789/0 conn(0x7f59c813dfd0 legacy=0x7f59c806e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:23.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59c77fe640 1 -- 172.21.15.135:0/2554779057 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 500632340 0 0) 0x7f59c813e3b0 con 0x7f59c813dfd0 2024-04-17T06:09:23.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.786+0000 7f59cd1b1640 1 -- 172.21.15.135:0/2554779057 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c80cd570 con 0x7f59c80cccf0 2024-04-17T06:09:23.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.787+0000 7f59cd1b1640 1 --2- 172.21.15.135:0/2554779057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c8140c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59c81417b0 tx=0x7f59c002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56d0a607dd7ccce server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:23.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.787+0000 7f59c77fe640 1 -- 172.21.15.135:0/2554779057 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c002cb20 con 0x7f59c80cccf0 2024-04-17T06:09:23.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.787+0000 7f59c77fe640 1 -- 172.21.15.135:0/2554779057 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c002ccc0 con 0x7f59c80cccf0 2024-04-17T06:09:23.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.787+0000 7f59c77fe640 1 -- 172.21.15.135:0/2554779057 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0034610 con 0x7f59c80cccf0 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.788+0000 7f59cec14640 1 -- 172.21.15.135:0/2554779057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 msgr2=0x7f59c8140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.788+0000 7f59cec14640 1 --2- 172.21.15.135:0/2554779057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c8140c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59c81417b0 tx=0x7f59c002bf80 comp rx=0 tx=0).stop 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.788+0000 7f59cec14640 1 -- 172.21.15.135:0/2554779057 shutdown_connections 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.788+0000 7f59cec14640 1 --2- 172.21.15.135:0/2554779057 >> v2:172.21.15.190:3300/0 conn(0x7f59c80cc840 0x7f59c8140710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.788+0000 7f59cec14640 1 --2- 172.21.15.135:0/2554779057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c8140c50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.788+0000 7f59cec14640 1 -- 172.21.15.135:0/2554779057 >> 172.21.15.135:0/2554779057 conn(0x7f59c80b2350 msgr2=0x7f59c80b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.788+0000 7f59cec14640 1 -- 172.21.15.135:0/2554779057 shutdown_connections 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.788+0000 7f59cec14640 1 -- 172.21.15.135:0/2554779057 wait complete. 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.789+0000 7f59cec14640 1 Processor -- start 2024-04-17T06:09:23.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.789+0000 7f59cec14640 1 -- start start 2024-04-17T06:09:23.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59cec14640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cc840 0x7f59c80b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:23.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59cec14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c806f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:23.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59cec14640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c813dfd0 0x7f59c806f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:23.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59cec14640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80cd850 con 0x7f59c80cccf0 2024-04-17T06:09:23.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59cec14640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80cf290 con 0x7f59c813dfd0 2024-04-17T06:09:23.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59cec14640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f59c8141400 con 0x7f59c80cc840 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c806f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c806f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53810/0 (socket says 172.21.15.135:53810) 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59c7fff640 1 -- 172.21.15.135:0/869404991 learned_addr learned my addr 172.21.15.135:0/869404991 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59cd1b1640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c813dfd0 0x7f59c806f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59cc9b0640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cc840 0x7f59c80b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59c7fff640 1 -- 172.21.15.135:0/869404991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cc840 msgr2=0x7f59c80b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59c7fff640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cc840 0x7f59c80b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59c7fff640 1 -- 172.21.15.135:0/869404991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c813dfd0 msgr2=0x7f59c806f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59c7fff640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c813dfd0 0x7f59c806f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.790+0000 7f59c7fff640 1 -- 172.21.15.135:0/869404991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c81f6c50 con 0x7f59c80cccf0 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.791+0000 7f59c7fff640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c806f0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b410 tx=0x7f59b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.791+0000 7f59c57fa640 1 -- 172.21.15.135:0/869404991 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4015020 con 0x7f59c80cccf0 2024-04-17T06:09:23.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.791+0000 7f59c57fa640 1 -- 172.21.15.135:0/869404991 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b4011850 con 0x7f59c80cccf0 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.791+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c81f2b90 con 0x7f59c80cccf0 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.791+0000 7f59c57fa640 1 -- 172.21.15.135:0/869404991 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4011c40 con 0x7f59c80cccf0 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.791+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c81f3150 con 0x7f59c80cccf0 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.792+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c81f29a0 con 0x7f59c80cccf0 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.792+0000 7f59c57fa640 1 -- 172.21.15.135:0/869404991 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59b4018020 con 0x7f59c80cccf0 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.793+0000 7f59c57fa640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59ac06c8f0 0x7f59ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.793+0000 7f59cc9b0640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59ac06c8f0 0x7f59ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.793+0000 7f59c57fa640 1 -- 172.21.15.135:0/869404991 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f59b408bcb0 con 0x7f59c80cccf0 2024-04-17T06:09:23.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.793+0000 7f59cc9b0640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59ac06c8f0 0x7f59ac06edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f59bc000f60 tx=0x7f59bc007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:23.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.798+0000 7f59c57fa640 1 -- 172.21.15.135:0/869404991 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59b401f020 con 0x7f59c80cccf0 2024-04-17T06:09:23.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:23.974+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f59c80749b0 con 0x7f59c80cccf0 2024-04-17T06:09:24.582 INFO:tasks.ceph.mds.d.smithi190.stderr:2024-04-17T06:09:24.584+0000 7f206f814640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T06:09:24.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.654+0000 7f59c57fa640 1 -- 172.21.15.135:0/869404991 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f59b4059770 con 0x7f59c80cccf0 2024-04-17T06:09:24.652 INFO:teuthology.orchestra.run.smithi135.stderr:set pool 4 allow_ec_overwrites to true 2024-04-17T06:09:24.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59ac06c8f0 msgr2=0x7f59ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:24.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59ac06c8f0 0x7f59ac06edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f59bc000f60 tx=0x7f59bc007d50 comp rx=0 tx=0).stop 2024-04-17T06:09:24.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 msgr2=0x7f59c806f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:24.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c806f0c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b410 tx=0x7f59b400b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 shutdown_connections 2024-04-17T06:09:24.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c813dfd0 0x7f59c806f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59ac06c8f0 0x7f59ac06edb0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f59bc000f60 tx=0x7f59bc007d50 comp rx=0 tx=0).stop 2024-04-17T06:09:24.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c80cccf0 0x7f59c806f0c0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59b400b410 tx=0x7f59b400b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 --2- 172.21.15.135:0/869404991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cc840 0x7f59c80b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.657+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 >> 172.21.15.135:0/869404991 conn(0x7f59c80b2350 msgr2=0x7f59c80b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:24.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.658+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 shutdown_connections 2024-04-17T06:09:24.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.658+0000 7f59cec14640 1 -- 172.21.15.135:0/869404991 wait complete. 2024-04-17T06:09:24.677 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-17T06:09:24.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.787+0000 7fcfd82c8640 1 Processor -- start 2024-04-17T06:09:24.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.787+0000 7fcfd82c8640 1 -- start start 2024-04-17T06:09:24.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd82c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:24.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd82c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcfd00d0490 0x7fcfd00c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:24.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd82c8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd00d0c90 con 0x7fcfd009a9f0 2024-04-17T06:09:24.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd82c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd00d0f70 con 0x7fcfd00d0490 2024-04-17T06:09:24.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd82c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd00d1250 con 0x7fcfd015cc60 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd6865640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcfd00d0490 0x7fcfd00c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd6865640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcfd00d0490 0x7fcfd00c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39726/0 (socket says 172.21.15.135:39726) 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd6865640 1 -- 172.21.15.135:0/2455724939 learned_addr learned my addr 172.21.15.135:0/2455724939 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd6064640 1 --1- 172.21.15.135:0/2455724939 >> v1:172.21.15.135:6789/0 conn(0x7fcfd009a9f0 0x7fcfd009add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35802/0 (socket says 172.21.15.135:35802) 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd5863640 1 --2- 172.21.15.135:0/2455724939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd6865640 1 -- 172.21.15.135:0/2455724939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 msgr2=0x7fcfd00c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.788+0000 7fcfd6865640 1 --2- 172.21.15.135:0/2455724939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.789+0000 7fcfd6865640 1 -- 172.21.15.135:0/2455724939 >> v1:172.21.15.135:6789/0 conn(0x7fcfd009a9f0 legacy=0x7fcfd009add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.789+0000 7fcfd6865640 1 -- 172.21.15.135:0/2455724939 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd00d1530 con 0x7fcfd00d0490 2024-04-17T06:09:24.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.789+0000 7fcfd5062640 1 -- 172.21.15.135:0/2455724939 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771923504 0 0) 0x7fcfd00d0c90 con 0x7fcfd009a9f0 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.789+0000 7fcfd5863640 1 --2- 172.21.15.135:0/2455724939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.789+0000 7fcfd6865640 1 --2- 172.21.15.135:0/2455724939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd00c6e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009e50 tx=0x7fcfc802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9df40aad8da105 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.789+0000 7fcfd5062640 1 -- 172.21.15.135:0/2455724939 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc802cb00 con 0x7fcfd00d0490 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.789+0000 7fcfd5062640 1 -- 172.21.15.135:0/2455724939 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc802cca0 con 0x7fcfd00d0490 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.789+0000 7fcfd5062640 1 -- 172.21.15.135:0/2455724939 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8034610 con 0x7fcfd00d0490 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.790+0000 7fcfd82c8640 1 -- 172.21.15.135:0/2455724939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 msgr2=0x7fcfd00c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.790+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/2455724939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd00c6e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009e50 tx=0x7fcfc802bf60 comp rx=0 tx=0).stop 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.790+0000 7fcfd82c8640 1 -- 172.21.15.135:0/2455724939 shutdown_connections 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.790+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/2455724939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.790+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/2455724939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd00c6e00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.790+0000 7fcfd82c8640 1 -- 172.21.15.135:0/2455724939 >> 172.21.15.135:0/2455724939 conn(0x7fcfd00b41a0 msgr2=0x7fcfd00b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:24.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.790+0000 7fcfd82c8640 1 -- 172.21.15.135:0/2455724939 shutdown_connections 2024-04-17T06:09:24.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.790+0000 7fcfd82c8640 1 -- 172.21.15.135:0/2455724939 wait complete. 2024-04-17T06:09:24.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.791+0000 7fcfd82c8640 1 Processor -- start 2024-04-17T06:09:24.793 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.791+0000 7fcfd82c8640 1 -- start start 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd82c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd009a9f0 0x7fcfd0153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd82c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd82c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd82c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd009bdf0 con 0x7fcfd00d0490 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd82c8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd00cca60 con 0x7fcfd009a9f0 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd82c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd009bf70 con 0x7fcfd015cc60 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd015d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39728/0 (socket says 172.21.15.135:39728) 2024-04-17T06:09:24.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 -- 172.21.15.135:0/428251400 learned_addr learned my addr 172.21.15.135:0/428251400 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd6865640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 -- 172.21.15.135:0/428251400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 msgr2=0x7fcfd00c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 -- 172.21.15.135:0/428251400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd009a9f0 msgr2=0x7fcfd0153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd009a9f0 0x7fcfd0153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 -- 172.21.15.135:0/428251400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd015ace0 con 0x7fcfd00d0490 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.792+0000 7fcfd5863640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd015d880 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c7c0 tx=0x7fcfc000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.793+0000 7fcfc6ffd640 1 -- 172.21.15.135:0/428251400 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0015070 con 0x7fcfd00d0490 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.793+0000 7fcfc6ffd640 1 -- 172.21.15.135:0/428251400 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc0007820 con 0x7fcfd00d0490 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.793+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd015afc0 con 0x7fcfd00d0490 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.793+0000 7fcfc6ffd640 1 -- 172.21.15.135:0/428251400 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0007bf0 con 0x7fcfd00d0490 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.793+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd00ca810 con 0x7fcfd00d0490 2024-04-17T06:09:24.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.793+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd0074940 con 0x7fcfd00d0490 2024-04-17T06:09:24.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.794+0000 7fcfc6ffd640 1 -- 172.21.15.135:0/428251400 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfc0018070 con 0x7fcfd00d0490 2024-04-17T06:09:24.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.794+0000 7fcfc6ffd640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcfb406c8f0 0x7fcfb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:24.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.795+0000 7fcfc6ffd640 1 -- 172.21.15.135:0/428251400 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fcfc008bc70 con 0x7fcfd00d0490 2024-04-17T06:09:24.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.795+0000 7fcfd6064640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcfb406c8f0 0x7fcfb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:24.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.795+0000 7fcfd6064640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcfb406c8f0 0x7fcfb406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc007fd0 tx=0x7fcfcc007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:24.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:24.799+0000 7fcfc6ffd640 1 -- 172.21.15.135:0/428251400 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfc0059e10 con 0x7fcfd00d0490 2024-04-17T06:09:25.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.016+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fcfd00cfc90 con 0x7fcfd00d0490 2024-04-17T06:09:25.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.655+0000 7fcfc6ffd640 1 -- 172.21.15.135:0/428251400 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fcfc0059710 con 0x7fcfd00d0490 2024-04-17T06:09:25.653 INFO:teuthology.orchestra.run.smithi135.stderr:added data pool 4 to fsmap 2024-04-17T06:09:25.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcfb406c8f0 msgr2=0x7fcfb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:25.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcfb406c8f0 0x7fcfb406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc007fd0 tx=0x7fcfcc007960 comp rx=0 tx=0).stop 2024-04-17T06:09:25.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 msgr2=0x7fcfd015d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:25.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd015d880 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c7c0 tx=0x7fcfc000cc80 comp rx=0 tx=0).stop 2024-04-17T06:09:25.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 shutdown_connections 2024-04-17T06:09:25.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfd015cc60 0x7fcfd00c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcfb406c8f0 0x7fcfb406edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcfd00d0490 0x7fcfd015d880 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 --2- 172.21.15.135:0/428251400 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfd009a9f0 0x7fcfd0153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 >> 172.21.15.135:0/428251400 conn(0x7fcfd00b41a0 msgr2=0x7fcfd00cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:25.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 shutdown_connections 2024-04-17T06:09:25.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.658+0000 7fcfd82c8640 1 -- 172.21.15.135:0/428251400 wait complete. 2024-04-17T06:09:25.670 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:09:25.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.781+0000 7f0bc6f15640 1 Processor -- start 2024-04-17T06:09:25.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.781+0000 7f0bc6f15640 1 -- start start 2024-04-17T06:09:25.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc6f15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc009a9f0 0x7f0bc009add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:25.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc6f15640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0bc00c78d0 0x7f0bc00cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc6f15640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc009b310 con 0x7f0bc015cc60 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc6f15640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc00d0c90 con 0x7f0bc00c78d0 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc6f15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc00d0f70 con 0x7f0bc009a9f0 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc4cb1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc009a9f0 0x7f0bc009add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc54b2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0bc00c78d0 0x7f0bc00cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc54b2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0bc00c78d0 0x7f0bc00cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:48812/0 (socket says 172.21.15.135:48812) 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc4cb1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc009a9f0 0x7f0bc009add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:60596/0 (socket says 172.21.15.135:60596) 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bb7fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f0bc015cc60 0x7f0bc00cd370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:58394/0 (socket says 172.21.15.135:58394) 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc54b2640 1 -- 172.21.15.135:0/1888469321 learned_addr learned my addr 172.21.15.135:0/1888469321 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc54b2640 1 -- 172.21.15.135:0/1888469321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc009a9f0 msgr2=0x7f0bc009add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc54b2640 1 --2- 172.21.15.135:0/1888469321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc009a9f0 0x7f0bc009add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc54b2640 1 -- 172.21.15.135:0/1888469321 >> v1:172.21.15.190:6789/0 conn(0x7f0bc015cc60 legacy=0x7f0bc00cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:25.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.782+0000 7f0bc54b2640 1 -- 172.21.15.135:0/1888469321 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc00d1250 con 0x7f0bc00c78d0 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.783+0000 7f0bc54b2640 1 --2- 172.21.15.135:0/1888469321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00cda80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009e50 tx=0x7f0bb802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ee7a55f78d2928 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.783+0000 7f0bb77fe640 1 -- 172.21.15.135:0/1888469321 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb802cb00 con 0x7f0bc00c78d0 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.783+0000 7f0bb77fe640 1 -- 172.21.15.135:0/1888469321 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb802cca0 con 0x7f0bc00c78d0 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.783+0000 7f0bb77fe640 1 -- 172.21.15.135:0/1888469321 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8034610 con 0x7f0bc00c78d0 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.784+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1888469321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc00c78d0 msgr2=0x7f0bc00cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.784+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1888469321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00cda80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009e50 tx=0x7f0bb802bf60 comp rx=0 tx=0).stop 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.784+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1888469321 shutdown_connections 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.784+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1888469321 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00cda80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.784+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1888469321 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc009a9f0 0x7f0bc009add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.784+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1888469321 >> 172.21.15.135:0/1888469321 conn(0x7f0bc00b41a0 msgr2=0x7f0bc00b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:25.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.784+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1888469321 shutdown_connections 2024-04-17T06:09:25.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.784+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1888469321 wait complete. 2024-04-17T06:09:25.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.785+0000 7f0bc6f15640 1 Processor -- start 2024-04-17T06:09:25.784 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.785+0000 7f0bc6f15640 1 -- start start 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bc6f15640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc009a9f0 0x7f0bc0153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bc6f15640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bc6f15640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc015cc60 0x7f0bc00b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bc6f15640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc00cca60 con 0x7f0bc00c78d0 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bc6f15640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc00c63a0 con 0x7f0bc009a9f0 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bc6f15640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc009bdf0 con 0x7f0bc015cc60 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00b7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39736/0 (socket says 172.21.15.135:39736) 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bb7fff640 1 -- 172.21.15.135:0/1630224916 learned_addr learned my addr 172.21.15.135:0/1630224916 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:25.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bc54b2640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc015cc60 0x7f0bc00b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bc4cb1640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc009a9f0 0x7f0bc0153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bb7fff640 1 -- 172.21.15.135:0/1630224916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc015cc60 msgr2=0x7f0bc00b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bb7fff640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc015cc60 0x7f0bc00b7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bb7fff640 1 -- 172.21.15.135:0/1630224916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc009a9f0 msgr2=0x7f0bc0153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bb7fff640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc009a9f0 0x7f0bc0153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.786+0000 7f0bb7fff640 1 -- 172.21.15.135:0/1630224916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc01f8d00 con 0x7f0bc00c78d0 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.787+0000 7f0bb7fff640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00b7940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c440 tx=0x7f0bb0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.787+0000 7f0bb57fa640 1 -- 172.21.15.135:0/1630224916 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00074e0 con 0x7f0bc00c78d0 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.787+0000 7f0bb57fa640 1 -- 172.21.15.135:0/1630224916 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb0007680 con 0x7f0bc00c78d0 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.787+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc01f7f40 con 0x7f0bc00c78d0 2024-04-17T06:09:25.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.787+0000 7f0bb57fa640 1 -- 172.21.15.135:0/1630224916 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0004a20 con 0x7f0bc00c78d0 2024-04-17T06:09:25.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.787+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc01f8500 con 0x7f0bc00c78d0 2024-04-17T06:09:25.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.788+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0074940 con 0x7f0bc00c78d0 2024-04-17T06:09:25.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.788+0000 7f0bb57fa640 1 -- 172.21.15.135:0/1630224916 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bb001a020 con 0x7f0bc00c78d0 2024-04-17T06:09:25.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.789+0000 7f0bb57fa640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0bac06c8f0 0x7f0bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:25.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.789+0000 7f0bc4cb1640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0bac06c8f0 0x7f0bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:25.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.789+0000 7f0bb57fa640 1 -- 172.21.15.135:0/1630224916 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0bb008b490 con 0x7f0bc00c78d0 2024-04-17T06:09:25.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.789+0000 7f0bc4cb1640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0bac06c8f0 0x7f0bac06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bc009b230 tx=0x7f0bbc009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:25.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:25.793+0000 7f0bb57fa640 1 -- 172.21.15.135:0/1630224916 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0bb0058fc0 con 0x7f0bc00c78d0 2024-04-17T06:09:26.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.009+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bc00c8b90 con 0x7f0bc00c78d0 2024-04-17T06:09:26.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.011+0000 7f0bb57fa640 1 -- 172.21.15.135:0/1630224916 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f0bb005d2c0 con 0x7f0bc00c78d0 2024-04-17T06:09:26.009 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:26.009 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:25.017414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:09:26.009 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 7 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0bac06c8f0 msgr2=0x7f0bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0bac06c8f0 0x7f0bac06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bc009b230 tx=0x7f0bbc009190 comp rx=0 tx=0).stop 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bc00c78d0 msgr2=0x7f0bc00b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00b7940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c440 tx=0x7f0bb0001e70 comp rx=0 tx=0).stop 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 shutdown_connections 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0bc015cc60 0x7f0bc00b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0bac06c8f0 0x7f0bac06edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bc00c78d0 0x7f0bc00b7940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 --2- 172.21.15.135:0/1630224916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0bc009a9f0 0x7f0bc0153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 >> 172.21.15.135:0/1630224916 conn(0x7f0bc00b41a0 msgr2=0x7f0bc00cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:26.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 shutdown_connections 2024-04-17T06:09:26.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.013+0000 7f0bc6f15640 1 -- 172.21.15.135:0/1630224916 wait complete. 2024-04-17T06:09:26.022 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:26.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.127+0000 7f64b21c3640 1 Processor -- start 2024-04-17T06:09:26.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.127+0000 7f64b21c3640 1 -- start start 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64b21c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64b21c3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64ac09bbe0 0x7f64ac09af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64b21c3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64ac06db80 con 0x7f64ac09b470 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64b21c3640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f64ac06de60 con 0x7f64ac09bbe0 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64b21c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac0cce30 con 0x7f64ac15ee30 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64aaffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f64ac09bbe0 0x7f64ac09af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64abfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f64ac09b470 0x7f64ac0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35814/0 (socket says 172.21.15.135:35814) 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64abfff640 1 -- 172.21.15.135:0/2488680569 learned_addr learned my addr 172.21.15.135:0/2488680569 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64ab7fe640 1 --2- 172.21.15.135:0/2488680569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64aaffd640 1 -- 172.21.15.135:0/2488680569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 msgr2=0x7f64ac09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:26.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64aaffd640 1 --2- 172.21.15.135:0/2488680569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64aaffd640 1 -- 172.21.15.135:0/2488680569 >> v1:172.21.15.135:6789/0 conn(0x7f64ac09b470 legacy=0x7f64ac0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64aaffd640 1 -- 172.21.15.135:0/2488680569 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0cd110 con 0x7f64ac09bbe0 2024-04-17T06:09:26.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.128+0000 7f64aa7fc640 1 -- 172.21.15.135:0/2488680569 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677761804 0 0) 0x7f64ac06db80 con 0x7f64ac09b470 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.129+0000 7f64aaffd640 1 --2- 172.21.15.135:0/2488680569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac09af30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64a00097a0 tx=0x7f64a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78b58455c75de0d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.129+0000 7f64aa7fc640 1 -- 172.21.15.135:0/2488680569 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0035020 con 0x7f64ac09bbe0 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.129+0000 7f64aa7fc640 1 -- 172.21.15.135:0/2488680569 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a0031830 con 0x7f64ac09bbe0 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.129+0000 7f64aa7fc640 1 -- 172.21.15.135:0/2488680569 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0031c00 con 0x7f64ac09bbe0 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.130+0000 7f64b21c3640 1 -- 172.21.15.135:0/2488680569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 msgr2=0x7f64ac09af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.130+0000 7f64b21c3640 1 --2- 172.21.15.135:0/2488680569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac09af30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64a00097a0 tx=0x7f64a002b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.130+0000 7f64b21c3640 1 -- 172.21.15.135:0/2488680569 shutdown_connections 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.130+0000 7f64b21c3640 1 --2- 172.21.15.135:0/2488680569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac09af30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.130+0000 7f64b21c3640 1 --2- 172.21.15.135:0/2488680569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.130+0000 7f64b21c3640 1 -- 172.21.15.135:0/2488680569 >> 172.21.15.135:0/2488680569 conn(0x7f64ac0b41a0 msgr2=0x7f64ac0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.130+0000 7f64b21c3640 1 -- 172.21.15.135:0/2488680569 shutdown_connections 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.130+0000 7f64b21c3640 1 -- 172.21.15.135:0/2488680569 wait complete. 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.131+0000 7f64b21c3640 1 Processor -- start 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.131+0000 7f64b21c3640 1 -- start start 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64b21c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac09b470 0x7f64ac155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64b21c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64b21c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64b21c3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac06d910 con 0x7f64ac09bbe0 2024-04-17T06:09:26.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64b21c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac06d790 con 0x7f64ac09b470 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64b21c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac0ce640 con 0x7f64ac15ee30 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64aaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64aaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac0b78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39744/0 (socket says 172.21.15.135:39744) 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64aaffd640 1 -- 172.21.15.135:0/3904350173 learned_addr learned my addr 172.21.15.135:0/3904350173 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64abfff640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64ab7fe640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac09b470 0x7f64ac155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64aaffd640 1 -- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 msgr2=0x7f64ac0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64aaffd640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac0c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64aaffd640 1 -- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac09b470 msgr2=0x7f64ac155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64aaffd640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac09b470 0x7f64ac155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64aaffd640 1 -- 172.21.15.135:0/3904350173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac0b6890 con 0x7f64ac09bbe0 2024-04-17T06:09:26.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.132+0000 7f64abfff640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:26.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.133+0000 7f64aaffd640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac0b78f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f64a00098c0 tx=0x7f64a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.133+0000 7f6493fff640 1 -- 172.21.15.135:0/3904350173 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0035040 con 0x7f64ac09bbe0 2024-04-17T06:09:26.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.133+0000 7f6493fff640 1 -- 172.21.15.135:0/3904350173 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a003b920 con 0x7f64ac09bbe0 2024-04-17T06:09:26.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.133+0000 7f6493fff640 1 -- 172.21.15.135:0/3904350173 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a003bd10 con 0x7f64ac09bbe0 2024-04-17T06:09:26.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.133+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac160970 con 0x7f64ac09bbe0 2024-04-17T06:09:26.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.133+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac0b6e10 con 0x7f64ac09bbe0 2024-04-17T06:09:26.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.134+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac074940 con 0x7f64ac09bbe0 2024-04-17T06:09:26.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.134+0000 7f6493fff640 1 -- 172.21.15.135:0/3904350173 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64a0038020 con 0x7f64ac09bbe0 2024-04-17T06:09:26.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.135+0000 7f6493fff640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.135+0000 7f64ab7fe640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.135+0000 7f6493fff640 1 -- 172.21.15.135:0/3904350173 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f64a00adfd0 con 0x7f64ac09bbe0 2024-04-17T06:09:26.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.135+0000 7f64ab7fe640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f649806c8f0 0x7f649806edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f649c009a30 tx=0x7f649c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.139+0000 7f6493fff640 1 -- 172.21.15.135:0/3904350173 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64a007ba70 con 0x7f64ac09bbe0 2024-04-17T06:09:26.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.319+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64ac071f50 con 0x7f64ac09bbe0 2024-04-17T06:09:26.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.320+0000 7f6493fff640 1 -- 172.21.15.135:0/3904350173 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f64a007fd70 con 0x7f64ac09bbe0 2024-04-17T06:09:26.319 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:26.319 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.322+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f649806c8f0 msgr2=0x7f649806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.322+0000 7f64b21c3640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f649806c8f0 0x7f649806edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f649c009a30 tx=0x7f649c008040 comp rx=0 tx=0).stop 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.322+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 msgr2=0x7f64ac0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.322+0000 7f64b21c3640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac0b78f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f64a00098c0 tx=0x7f64a00046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.323+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 shutdown_connections 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.323+0000 7f64b21c3640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac15ee30 0x7f64ac0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.323+0000 7f64b21c3640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f649806c8f0 0x7f649806edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.323+0000 7f64b21c3640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64ac09bbe0 0x7f64ac0b78f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.323+0000 7f64b21c3640 1 --2- 172.21.15.135:0/3904350173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac09b470 0x7f64ac155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.323+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 >> 172.21.15.135:0/3904350173 conn(0x7f64ac0b41a0 msgr2=0x7f64ac070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.323+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 shutdown_connections 2024-04-17T06:09:26.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.323+0000 7f64b21c3640 1 -- 172.21.15.135:0/3904350173 wait complete. 2024-04-17T06:09:26.332 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:26.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.438+0000 7f9aaf531640 1 Processor -- start 2024-04-17T06:09:26.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.438+0000 7f9aaf531640 1 -- start start 2024-04-17T06:09:26.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aaf531640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa815f800 0x7f9aa8070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aaf531640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9aa80d0f20 0x7f9aa80cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aaf531640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aa806bb30 con 0x7f9aa8070c90 2024-04-17T06:09:26.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aaf531640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9aa806c8f0 con 0x7f9aa80d0f20 2024-04-17T06:09:26.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aaf531640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa806f070 con 0x7f9aa815f800 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aad2cd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa815f800 0x7f9aa8070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aadace640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9aa80d0f20 0x7f9aa80cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aad2cd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa815f800 0x7f9aa8070750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:60632/0 (socket says 172.21.15.135:60632) 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aad2cd640 1 -- 172.21.15.135:0/1484621971 learned_addr learned my addr 172.21.15.135:0/1484621971 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aad2cd640 1 -- 172.21.15.135:0/1484621971 >> v2:172.21.15.190:3300/0 conn(0x7f9aa80d0f20 msgr2=0x7f9aa80cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aad2cd640 1 --2- 172.21.15.135:0/1484621971 >> v2:172.21.15.190:3300/0 conn(0x7f9aa80d0f20 0x7f9aa80cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aad2cd640 1 -- 172.21.15.135:0/1484621971 >> v1:172.21.15.190:6789/0 conn(0x7f9aa8070c90 legacy=0x7f9aa80d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aad2cd640 1 -- 172.21.15.135:0/1484621971 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa806be10 con 0x7f9aa815f800 2024-04-17T06:09:26.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.439+0000 7f9aadace640 1 --2- 172.21.15.135:0/1484621971 >> v2:172.21.15.190:3300/0 conn(0x7f9aa80d0f20 0x7f9aa80cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:26.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.440+0000 7f9aad2cd640 1 --2- 172.21.15.135:0/1484621971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa815f800 0x7f9aa8070750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009870 tx=0x7f9aa402ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240bcb33f465014e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.440+0000 7f9a9ffff640 1 -- 172.21.15.135:0/1484621971 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa402f840 con 0x7f9aa815f800 2024-04-17T06:09:26.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.440+0000 7f9a9ffff640 1 -- 172.21.15.135:0/1484621971 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa402f9e0 con 0x7f9aa815f800 2024-04-17T06:09:26.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.440+0000 7f9a9ffff640 1 -- 172.21.15.135:0/1484621971 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa402fdb0 con 0x7f9aa815f800 2024-04-17T06:09:26.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.441+0000 7f9aaf531640 1 -- 172.21.15.135:0/1484621971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa815f800 msgr2=0x7f9aa8070750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.441+0000 7f9aaf531640 1 --2- 172.21.15.135:0/1484621971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa815f800 0x7f9aa8070750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009870 tx=0x7f9aa402ba20 comp rx=0 tx=0).stop 2024-04-17T06:09:26.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.441+0000 7f9aaf531640 1 -- 172.21.15.135:0/1484621971 shutdown_connections 2024-04-17T06:09:26.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.441+0000 7f9aaf531640 1 --2- 172.21.15.135:0/1484621971 >> v2:172.21.15.190:3300/0 conn(0x7f9aa80d0f20 0x7f9aa80cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.441+0000 7f9aaf531640 1 --2- 172.21.15.135:0/1484621971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa815f800 0x7f9aa8070750 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.441+0000 7f9aaf531640 1 -- 172.21.15.135:0/1484621971 >> 172.21.15.135:0/1484621971 conn(0x7f9aa809a9f0 msgr2=0x7f9aa80b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:26.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.441+0000 7f9aaf531640 1 -- 172.21.15.135:0/1484621971 shutdown_connections 2024-04-17T06:09:26.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.441+0000 7f9aaf531640 1 -- 172.21.15.135:0/1484621971 wait complete. 2024-04-17T06:09:26.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.442+0000 7f9aaf531640 1 Processor -- start 2024-04-17T06:09:26.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.442+0000 7f9aaf531640 1 -- start start 2024-04-17T06:09:26.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aaf531640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa8070c90 0x7f9aa80b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aaf531640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa80d0f20 0x7f9aa815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aaf531640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa815f800 0x7f9aa815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aaf531640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa815fbe0 con 0x7f9aa8070c90 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aaf531640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa806c460 con 0x7f9aa815f800 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aaf531640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa815fd60 con 0x7f9aa80d0f20 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa8070c90 0x7f9aa80b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa8070c90 0x7f9aa80b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39750/0 (socket says 172.21.15.135:39750) 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 -- 172.21.15.135:0/3153503093 learned_addr learned my addr 172.21.15.135:0/3153503093 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aadace640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa815f800 0x7f9aa815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aacacc640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa80d0f20 0x7f9aa815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 -- 172.21.15.135:0/3153503093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa80d0f20 msgr2=0x7f9aa815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa80d0f20 0x7f9aa815d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 -- 172.21.15.135:0/3153503093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa815f800 msgr2=0x7f9aa815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa815f800 0x7f9aa815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 -- 172.21.15.135:0/3153503093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa80b4c20 con 0x7f9aa8070c90 2024-04-17T06:09:26.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aad2cd640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa8070c90 0x7f9aa80b41a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa402ba60 tx=0x7f9aa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9a9dffb640 1 -- 172.21.15.135:0/3153503093 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4047070 con 0x7f9aa8070c90 2024-04-17T06:09:26.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9a9dffb640 1 -- 172.21.15.135:0/3153503093 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa4037c40 con 0x7f9aa8070c90 2024-04-17T06:09:26.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa80b4f00 con 0x7f9aa8070c90 2024-04-17T06:09:26.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.443+0000 7f9a9dffb640 1 -- 172.21.15.135:0/3153503093 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40423f0 con 0x7f9aa8070c90 2024-04-17T06:09:26.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.444+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa815a4d0 con 0x7f9aa8070c90 2024-04-17T06:09:26.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.445+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa8074940 con 0x7f9aa8070c90 2024-04-17T06:09:26.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.445+0000 7f9a9dffb640 1 -- 172.21.15.135:0/3153503093 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9aa404a070 con 0x7f9aa8070c90 2024-04-17T06:09:26.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.445+0000 7f9a9dffb640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.446+0000 7f9a9dffb640 1 -- 172.21.15.135:0/3153503093 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9aa40adbb0 con 0x7f9aa8070c90 2024-04-17T06:09:26.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.446+0000 7f9aacacc640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.446+0000 7f9aacacc640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80b5ef0 tx=0x7f9a98009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.447 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.450+0000 7f9a9dffb640 1 -- 172.21.15.135:0/3153503093 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9aa4033030 con 0x7f9aa8070c90 2024-04-17T06:09:26.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.626+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9aa8073be0 con 0x7f9aa8070c90 2024-04-17T06:09:26.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.627+0000 7f9a9dffb640 1 -- 172.21.15.135:0/3153503093 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f9aa407b6e0 con 0x7f9aa8070c90 2024-04-17T06:09:26.626 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:26.626 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:26.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a8c06c8f0 msgr2=0x7f9a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80b5ef0 tx=0x7f9a98009040 comp rx=0 tx=0).stop 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa8070c90 msgr2=0x7f9aa80b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa8070c90 0x7f9aa80b41a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa402ba60 tx=0x7f9aa4004730 comp rx=0 tx=0).stop 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 shutdown_connections 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9aa815f800 0x7f9aa815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9aa80d0f20 0x7f9aa815d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a8c06c8f0 0x7f9a8c06edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 --2- 172.21.15.135:0/3153503093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa8070c90 0x7f9aa80b41a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 >> 172.21.15.135:0/3153503093 conn(0x7f9aa809a9f0 msgr2=0x7f9aa80b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 shutdown_connections 2024-04-17T06:09:26.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.629+0000 7f9aaf531640 1 -- 172.21.15.135:0/3153503093 wait complete. 2024-04-17T06:09:26.653 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:09:26.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.759+0000 7f436a959640 1 Processor -- start 2024-04-17T06:09:26.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.759+0000 7f436a959640 1 -- start start 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f436a959640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f436415ee30 0x7f436409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f436a959640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f43640cd630 0x7f43640cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f436a959640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436406db80 con 0x7f436409bbe0 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f436a959640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f436406de60 con 0x7f436415ee30 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f436a959640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f43640cce30 con 0x7f43640cd630 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f4363fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f436415ee30 0x7f436409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f435bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f436409bbe0 0x7f436409af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35816/0 (socket says 172.21.15.135:35816) 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f4363fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f436415ee30 0x7f436409a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39756/0 (socket says 172.21.15.135:39756) 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f435bfff640 1 -- 172.21.15.135:0/3555919573 learned_addr learned my addr 172.21.15.135:0/3555919573 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f4368ef6640 1 --2- 172.21.15.135:0/3555919573 >> v2:172.21.15.190:3300/0 conn(0x7f43640cd630 0x7f43640cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f4363fff640 1 -- 172.21.15.135:0/3555919573 >> v2:172.21.15.190:3300/0 conn(0x7f43640cd630 msgr2=0x7f43640cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:26.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f4363fff640 1 --2- 172.21.15.135:0/3555919573 >> v2:172.21.15.190:3300/0 conn(0x7f43640cd630 0x7f43640cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f4363fff640 1 -- 172.21.15.135:0/3555919573 >> v1:172.21.15.135:6789/0 conn(0x7f436409bbe0 legacy=0x7f436409af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f43637fe640 1 -- 172.21.15.135:0/3555919573 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724761226 0 0) 0x7f436406db80 con 0x7f436409bbe0 2024-04-17T06:09:26.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.760+0000 7f4363fff640 1 -- 172.21.15.135:0/3555919573 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43640cd110 con 0x7f436415ee30 2024-04-17T06:09:26.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.761+0000 7f4363fff640 1 --2- 172.21.15.135:0/3555919573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436415ee30 0x7f436409a9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f436406cb30 tx=0x7f435402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286ef38276aed7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.761+0000 7f43637fe640 1 -- 172.21.15.135:0/3555919573 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354035020 con 0x7f436415ee30 2024-04-17T06:09:26.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.761+0000 7f43637fe640 1 -- 172.21.15.135:0/3555919573 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4354031830 con 0x7f436415ee30 2024-04-17T06:09:26.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.761+0000 7f43637fe640 1 -- 172.21.15.135:0/3555919573 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354031c00 con 0x7f436415ee30 2024-04-17T06:09:26.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.762+0000 7f436a959640 1 -- 172.21.15.135:0/3555919573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436415ee30 msgr2=0x7f436409a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.762+0000 7f436a959640 1 --2- 172.21.15.135:0/3555919573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436415ee30 0x7f436409a9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f436406cb30 tx=0x7f435402b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.762+0000 7f436a959640 1 -- 172.21.15.135:0/3555919573 shutdown_connections 2024-04-17T06:09:26.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.762+0000 7f436a959640 1 --2- 172.21.15.135:0/3555919573 >> v2:172.21.15.190:3300/0 conn(0x7f43640cd630 0x7f43640cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.762+0000 7f436a959640 1 --2- 172.21.15.135:0/3555919573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436415ee30 0x7f436409a9f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.762+0000 7f436a959640 1 -- 172.21.15.135:0/3555919573 >> 172.21.15.135:0/3555919573 conn(0x7f43640b41a0 msgr2=0x7f43640b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.762+0000 7f436a959640 1 -- 172.21.15.135:0/3555919573 shutdown_connections 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.762+0000 7f436a959640 1 -- 172.21.15.135:0/3555919573 wait complete. 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.763+0000 7f436a959640 1 Processor -- start 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.763+0000 7f436a959640 1 -- start start 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f436a959640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436409bbe0 0x7f4364158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f436a959640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43640cd630 0x7f43640cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f436a959640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f436415ee30 0x7f436415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f436a959640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f436406d910 con 0x7f436409bbe0 2024-04-17T06:09:26.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f436a959640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f436406d790 con 0x7f43640cd630 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f436a959640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f436409b640 con 0x7f436415ee30 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436409bbe0 0x7f4364158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436409bbe0 0x7f4364158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39770/0 (socket says 172.21.15.135:39770) 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 -- 172.21.15.135:0/3508706302 learned_addr learned my addr 172.21.15.135:0/3508706302 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f435bfff640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43640cd630 0x7f43640cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4368ef6640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f436415ee30 0x7f436415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 -- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f436415ee30 msgr2=0x7f436415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f436415ee30 0x7f436415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 -- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43640cd630 msgr2=0x7f43640cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43640cd630 0x7f43640cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 -- 172.21.15.135:0/3508706302 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43640b5220 con 0x7f436409bbe0 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4368ef6640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f436415ee30 0x7f436415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f4363fff640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436409bbe0 0x7f4364158310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4354009890 tx=0x7f43540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f43617fa640 1 -- 172.21.15.135:0/3508706302 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354040070 con 0x7f436409bbe0 2024-04-17T06:09:26.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f43617fa640 1 -- 172.21.15.135:0/3508706302 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435403e070 con 0x7f436409bbe0 2024-04-17T06:09:26.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43640b5490 con 0x7f436409bbe0 2024-04-17T06:09:26.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f43617fa640 1 -- 172.21.15.135:0/3508706302 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354035040 con 0x7f436409bbe0 2024-04-17T06:09:26.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.764+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43640b5a50 con 0x7f436409bbe0 2024-04-17T06:09:26.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.765+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4364074940 con 0x7f436409bbe0 2024-04-17T06:09:26.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.766+0000 7f43617fa640 1 -- 172.21.15.135:0/3508706302 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4354038020 con 0x7f436409bbe0 2024-04-17T06:09:26.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.766+0000 7f43617fa640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f434406c8f0 0x7f434406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:26.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.766+0000 7f435bfff640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f434406c8f0 0x7f434406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:26.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.766+0000 7f43617fa640 1 -- 172.21.15.135:0/3508706302 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f43540ad810 con 0x7f436409bbe0 2024-04-17T06:09:26.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.767+0000 7f435bfff640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f434406c8f0 0x7f434406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43640c93e0 tx=0x7f434c007b50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:26.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.771+0000 7f43617fa640 1 -- 172.21.15.135:0/3508706302 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f435407b990 con 0x7f436409bbe0 2024-04-17T06:09:26.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.983+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43640d0400 con 0x7f436409bbe0 2024-04-17T06:09:26.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.983+0000 7f43617fa640 1 -- 172.21.15.135:0/3508706302 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f43540abe20 con 0x7f436409bbe0 2024-04-17T06:09:26.981 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:26.982 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:25.017414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:09:26.982 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 7 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f434406c8f0 msgr2=0x7f434406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f434406c8f0 0x7f434406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43640c93e0 tx=0x7f434c007b50 comp rx=0 tx=0).stop 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436409bbe0 msgr2=0x7f4364158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436409bbe0 0x7f4364158310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4354009890 tx=0x7f43540046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 shutdown_connections 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f436415ee30 0x7f436415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43640cd630 0x7f43640cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f434406c8f0 0x7f434406edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 --2- 172.21.15.135:0/3508706302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f436409bbe0 0x7f4364158310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:26.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 >> 172.21.15.135:0/3508706302 conn(0x7f43640b41a0 msgr2=0x7f4364070150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:26.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.985+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 shutdown_connections 2024-04-17T06:09:26.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:26.986+0000 7f436a959640 1 -- 172.21.15.135:0/3508706302 wait complete. 2024-04-17T06:09:26.995 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:27.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.100+0000 7f81f5a0d640 1 Processor -- start 2024-04-17T06:09:27.098 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.100+0000 7f81f5a0d640 1 -- start start 2024-04-17T06:09:27.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81f5a0d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f81f015ec00 0x7f81f00d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81f5a0d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81f00d19e0 0x7f81f00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81f5a0d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f00d3330 con 0x7f81f00d43d0 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81f5a0d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f81f015efe0 con 0x7f81f015ec00 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81f5a0d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f81f00d3f40 con 0x7f81f00d19e0 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81eeffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f81f015ec00 0x7f81f00d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81eeffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f81f015ec00 0x7f81f00d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39780/0 (socket says 172.21.15.135:39780) 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81eeffd640 1 -- 172.21.15.135:0/2123032123 learned_addr learned my addr 172.21.15.135:0/2123032123 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81ef7fe640 1 --2- 172.21.15.135:0/2123032123 >> v2:172.21.15.190:3300/0 conn(0x7f81f00d19e0 0x7f81f00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81eeffd640 1 -- 172.21.15.135:0/2123032123 >> v2:172.21.15.190:3300/0 conn(0x7f81f00d19e0 msgr2=0x7f81f00b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81eeffd640 1 --2- 172.21.15.135:0/2123032123 >> v2:172.21.15.190:3300/0 conn(0x7f81f00d19e0 0x7f81f00b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81eeffd640 1 -- 172.21.15.135:0/2123032123 >> v1:172.21.15.135:6789/0 conn(0x7f81f00d43d0 legacy=0x7f81f00d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81ee7fc640 1 -- 172.21.15.135:0/2123032123 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 646281946 0 0) 0x7f81f00d3330 con 0x7f81f00d43d0 2024-04-17T06:09:27.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81eeffd640 1 -- 172.21.15.135:0/2123032123 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f00d3610 con 0x7f81f015ec00 2024-04-17T06:09:27.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.101+0000 7f81ef7fe640 1 --2- 172.21.15.135:0/2123032123 >> v2:172.21.15.190:3300/0 conn(0x7f81f00d19e0 0x7f81f00b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:27.101 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.102+0000 7f81eeffd640 1 --2- 172.21.15.135:0/2123032123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00d2990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f81f006dab0 tx=0x7f81e002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8344d6422b8d4f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.102+0000 7f81ee7fc640 1 -- 172.21.15.135:0/2123032123 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e002f850 con 0x7f81f015ec00 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.102+0000 7f81ee7fc640 1 -- 172.21.15.135:0/2123032123 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e002f9f0 con 0x7f81f015ec00 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.102+0000 7f81ee7fc640 1 -- 172.21.15.135:0/2123032123 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e002fdc0 con 0x7f81f015ec00 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.103+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2123032123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 msgr2=0x7f81f00d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.103+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2123032123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00d2990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f81f006dab0 tx=0x7f81e002ba30 comp rx=0 tx=0).stop 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.103+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2123032123 shutdown_connections 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.103+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2123032123 >> v2:172.21.15.190:3300/0 conn(0x7f81f00d19e0 0x7f81f00b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.103+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2123032123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00d2990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.103+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2123032123 >> 172.21.15.135:0/2123032123 conn(0x7f81f006cc90 msgr2=0x7f81f006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.103+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2123032123 shutdown_connections 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.103+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2123032123 wait complete. 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.104+0000 7f81f5a0d640 1 Processor -- start 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.104+0000 7f81f5a0d640 1 -- start start 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81f5a0d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f00d19e0 0x7f81f0151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81f5a0d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f00d43d0 0x7f81f00cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81f5a0d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81f5a0d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81f00d2ed0 con 0x7f81f015ec00 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81f5a0d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f81f00d3bd0 con 0x7f81f00d19e0 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81f5a0d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f81f00d3050 con 0x7f81f00d43d0 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39794/0 (socket says 172.21.15.135:39794) 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 -- 172.21.15.135:0/2282784667 learned_addr learned my addr 172.21.15.135:0/2282784667 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81e7fff640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f00d43d0 0x7f81f00cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81eeffd640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f00d19e0 0x7f81f0151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 -- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f00d43d0 msgr2=0x7f81f00cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f00d43d0 0x7f81f00cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 -- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f00d19e0 msgr2=0x7f81f0151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f00d19e0 0x7f81f0151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.106 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 -- 172.21.15.135:0/2282784667 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f00cd270 con 0x7f81f015ec00 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81eeffd640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f00d19e0 0x7f81f0151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81e7fff640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f00d43d0 0x7f81f00cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ef7fe640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00ce3e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f81e8007e40 tx=0x7f81e800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ecff9640 1 -- 172.21.15.135:0/2282784667 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e801a070 con 0x7f81f015ec00 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ecff9640 1 -- 172.21.15.135:0/2282784667 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e8015600 con 0x7f81f015ec00 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81ecff9640 1 -- 172.21.15.135:0/2282784667 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80159f0 con 0x7f81f015ec00 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f00c85c0 con 0x7f81f015ec00 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.105+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f00cc510 con 0x7f81f015ec00 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.106+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f0074940 con 0x7f81f015ec00 2024-04-17T06:09:27.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.107+0000 7f81ecff9640 1 -- 172.21.15.135:0/2282784667 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81e801d070 con 0x7f81f015ec00 2024-04-17T06:09:27.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.107+0000 7f81ecff9640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81d006c8f0 0x7f81d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.107+0000 7f81eeffd640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81d006c8f0 0x7f81d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.107+0000 7f81ecff9640 1 -- 172.21.15.135:0/2282784667 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f81e808c190 con 0x7f81f015ec00 2024-04-17T06:09:27.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.108+0000 7f81eeffd640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81d006c8f0 0x7f81d006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81e0036000 tx=0x7f81e0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.112+0000 7f81ecff9640 1 -- 172.21.15.135:0/2282784667 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81e8010070 con 0x7f81f015ec00 2024-04-17T06:09:27.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.289+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81f00b5380 con 0x7f81f015ec00 2024-04-17T06:09:27.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.290+0000 7f81ecff9640 1 -- 172.21.15.135:0/2282784667 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f81e805a280 con 0x7f81f015ec00 2024-04-17T06:09:27.289 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:27.289 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81d006c8f0 msgr2=0x7f81d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81d006c8f0 0x7f81d006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81e0036000 tx=0x7f81e0008030 comp rx=0 tx=0).stop 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 msgr2=0x7f81f00ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00ce3e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f81e8007e40 tx=0x7f81e800d4a0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 shutdown_connections 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81d006c8f0 0x7f81d006edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81f015ec00 0x7f81f00ce3e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81f00d43d0 0x7f81f00cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 --2- 172.21.15.135:0/2282784667 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81f00d19e0 0x7f81f0151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.292+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 >> 172.21.15.135:0/2282784667 conn(0x7f81f006cc90 msgr2=0x7f81f00b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.293+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 shutdown_connections 2024-04-17T06:09:27.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.293+0000 7f81f5a0d640 1 -- 172.21.15.135:0/2282784667 wait complete. 2024-04-17T06:09:27.705 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T06:09:19.967271+0000', 'modified': '2024-04-17T06:09:25.017414+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/3795516465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 3795516465}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 3795516465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-17T06:09:27.705 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T06:09:27.705 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-17T06:09:27.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.813+0000 7f2ae2cd3640 1 Processor -- start 2024-04-17T06:09:27.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.813+0000 7f2ae2cd3640 1 -- start start 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae2cd3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc0d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae2cd3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2adc0d0490 0x7f2adc09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae2cd3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2adc09af30 0x7f2adc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae2cd3640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2adc15d040 con 0x7f2adc09af30 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae2cd3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f2adc0d13c0 con 0x7f2adc0d0490 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae2cd3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc0b1d20 con 0x7f2adc15cc60 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae1270640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2adc09af30 0x7f2adc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae1270640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2adc09af30 0x7f2adc0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39806/0 (socket says 172.21.15.135:39806) 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae1270640 1 -- 172.21.15.135:0/1077532849 learned_addr learned my addr 172.21.15.135:0/1077532849 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae1270640 1 -- 172.21.15.135:0/1077532849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 msgr2=0x7f2adc0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1077532849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc0d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1077532849 >> v2:172.21.15.190:3300/0 conn(0x7f2adc0d0490 0x7f2adc09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1077532849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc0d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae1270640 1 -- 172.21.15.135:0/1077532849 >> v2:172.21.15.190:3300/0 conn(0x7f2adc0d0490 msgr2=0x7f2adc09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1077532849 >> v2:172.21.15.190:3300/0 conn(0x7f2adc0d0490 0x7f2adc09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae1270640 1 -- 172.21.15.135:0/1077532849 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc0b2000 con 0x7f2adc09af30 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1077532849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc0d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.814+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1077532849 >> v2:172.21.15.190:3300/0 conn(0x7f2adc0d0490 0x7f2adc09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.815+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1077532849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc09af30 0x7f2adc0cd300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2adc069720 tx=0x7f2ad402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99eeecd7c26fb7e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.815+0000 7f2ad37fe640 1 -- 172.21.15.135:0/1077532849 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4035020 con 0x7f2adc09af30 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.815+0000 7f2ad37fe640 1 -- 172.21.15.135:0/1077532849 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad4031870 con 0x7f2adc09af30 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.815+0000 7f2ad37fe640 1 -- 172.21.15.135:0/1077532849 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4031c40 con 0x7f2adc09af30 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/1077532849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc09af30 msgr2=0x7f2adc0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/1077532849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc09af30 0x7f2adc0cd300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2adc069720 tx=0x7f2ad402b910 comp rx=0 tx=0).stop 2024-04-17T06:09:27.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/1077532849 shutdown_connections 2024-04-17T06:09:27.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/1077532849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc09af30 0x7f2adc0cd300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/1077532849 >> v2:172.21.15.190:3300/0 conn(0x7f2adc0d0490 0x7f2adc09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/1077532849 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc0d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/1077532849 >> 172.21.15.135:0/1077532849 conn(0x7f2adc0b41a0 msgr2=0x7f2adc0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/1077532849 shutdown_connections 2024-04-17T06:09:27.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.816+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/1077532849 wait complete. 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.817+0000 7f2ae2cd3640 1 Processor -- start 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.817+0000 7f2ae2cd3640 1 -- start start 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae2cd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc09af30 0x7f2adc0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae2cd3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc0d0490 0x7f2adc15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae2cd3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae2cd3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0d16a0 con 0x7f2adc0d0490 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae2cd3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0d10d0 con 0x7f2adc09af30 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae2cd3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc0d1820 con 0x7f2adc15cc60 2024-04-17T06:09:27.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc0d0490 0x7f2adc15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc0d0490 0x7f2adc15dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39816/0 (socket says 172.21.15.135:39816) 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 -- 172.21.15.135:0/3793208769 learned_addr learned my addr 172.21.15.135:0/3793208769 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc09af30 0x7f2adc0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae1270640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 -- 172.21.15.135:0/3793208769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 msgr2=0x7f2adc151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 -- 172.21.15.135:0/3793208769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc09af30 msgr2=0x7f2adc0c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc09af30 0x7f2adc0c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 -- 172.21.15.135:0/3793208769 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc0b6130 con 0x7f2adc0d0490 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc0d0490 0x7f2adc15dc30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c7c0 tx=0x7f2acc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad17fa640 1 -- 172.21.15.135:0/3793208769 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc015070 con 0x7f2adc0d0490 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad17fa640 1 -- 172.21.15.135:0/3793208769 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc007a30 con 0x7f2adc0d0490 2024-04-17T06:09:27.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ad17fa640 1 -- 172.21.15.135:0/3793208769 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc007e00 con 0x7f2adc0d0490 2024-04-17T06:09:27.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.818+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/3793208769 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc15e330 con 0x7f2adc0d0490 2024-04-17T06:09:27.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.819+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/3793208769 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc15e8f0 con 0x7f2adc0d0490 2024-04-17T06:09:27.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.820+0000 7f2ad17fa640 1 -- 172.21.15.135:0/3793208769 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2acc018070 con 0x7f2adc0d0490 2024-04-17T06:09:27.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.821+0000 7f2ad17fa640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2ac806c8f0 0x7f2ac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.821+0000 7f2ad17fa640 1 -- 172.21.15.135:0/3793208769 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2acc08bc30 con 0x7f2adc0d0490 2024-04-17T06:09:27.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.821+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2ac806c8f0 0x7f2ac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.821+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2ac806c8f0 0x7f2ac806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800d620 tx=0x7f2ad8007810 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.822+0000 7f2ae5996740 1 Processor -- start 2024-04-17T06:09:27.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.822+0000 7f2ae5996740 1 -- start start 2024-04-17T06:09:27.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.822+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face90ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.822+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face93b1f0 0x55face93dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.822+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face90e2f0 0x55face93e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.822+0000 7f2ae5996740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55face8789c0 con 0x55face90e2f0 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.822+0000 7f2ae5996740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55face81f650 con 0x55face93ae10 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.822+0000 7f2ae5996740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55face874980 con 0x55face93b1f0 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face90e2f0 0x55face93e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face90e2f0 0x55face93e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39828/0 (socket says 172.21.15.135:39828) 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 -- 172.21.15.135:0/1684365759 learned_addr learned my addr 172.21.15.135:0/1684365759 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face93b1f0 0x55face93dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face90ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 -- 172.21.15.135:0/1684365759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face93b1f0 msgr2=0x55face93dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face93b1f0 0x55face93dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 -- 172.21.15.135:0/1684365759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 msgr2=0x55face90ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face90ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 -- 172.21.15.135:0/1684365759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55face58b2d0 con 0x55face90e2f0 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face90e2f0 0x55face93e0e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40090b0 tx=0x7f2ad4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1264e80c73e9cda2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ac27fc640 1 -- 172.21.15.135:0/1684365759 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad404a070 con 0x55face90e2f0 2024-04-17T06:09:27.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ac27fc640 1 -- 172.21.15.135:0/1684365759 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad4031c40 con 0x55face90e2f0 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.823+0000 7f2ac27fc640 1 -- 172.21.15.135:0/1684365759 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4035040 con 0x55face90e2f0 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 -- 172.21.15.135:0/1684365759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face90e2f0 msgr2=0x55face93e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face90e2f0 0x55face93e0e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40090b0 tx=0x7f2ad4008030 comp rx=0 tx=0).stop 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 -- 172.21.15.135:0/1684365759 shutdown_connections 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face90e2f0 0x55face93e0e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face93b1f0 0x55face93dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1684365759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face90ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 -- 172.21.15.135:0/1684365759 >> 172.21.15.135:0/1684365759 conn(0x55face910020 msgr2=0x55face958ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 -- 172.21.15.135:0/1684365759 shutdown_connections 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 -- 172.21.15.135:0/1684365759 wait complete. 2024-04-17T06:09:27.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.824+0000 7f2ae5996740 1 -- start start 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae5996740 1 Processor -- start 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face936370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93b1f0 0x55face96c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 0x55face96c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae5996740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55face9125c0 con 0x55face93ae10 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae5996740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55face832a10 con 0x55face93b1f0 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae5996740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55face9119b0 con 0x55face934050 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae5996740 10 client.0 _ll_register_callbacks cb 0x55face86e250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae0a6f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face936370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae0a6f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face936370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39840/0 (socket says 172.21.15.135:39840) 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae0a6f640 1 -- 172.21.15.135:0/1100082952 learned_addr learned my addr 172.21.15.135:0/1100082952 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 0x55face96c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.825+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93b1f0 0x55face96c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae0a6f640 1 -- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 msgr2=0x55face96c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 0x55face96c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae0a6f640 1 -- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93b1f0 msgr2=0x55face96c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93b1f0 0x55face96c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae0a6f640 1 -- 172.21.15.135:0/1100082952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55face6556b0 con 0x55face93ae10 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face936370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800cec0 tx=0x7f2ad8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4073afc8d465b22a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2abb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1100082952 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8018600 con 0x55face93ae10 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1100082952 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ad80187a0 con 0x55face93ae10 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 1 -- 172.21.15.135:0/1100082952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad4031c40 con 0x55face93ae10 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1100082952 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2ad80189a0 con 0x55face93ae10 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 10 client.4434 fetch_fsmap learned FSMap version 7 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 1 -- 172.21.15.135:0/1100082952 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55face93bde0 con 0x55face93ae10 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1100082952 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad4031c40 con 0x55face93ae10 2024-04-17T06:09:27.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1100082952 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f2ad802d050 con 0x55face93ae10 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 10 client.4434 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 20 client.4434 populate_metadata read hostname 'smithi135' 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55face9243c0 0x55face9247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 4 client.4434 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.826+0000 7f2ae5996740 1 -- 172.21.15.135:0/1100082952 --> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55face8863e0 con 0x55face9243c0 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.827+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55face9243c0 0x55face9247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.827+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55face9243c0 0x55face9247a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd169bffda607adc server_cookie=5471574ce8710079 in_seq=0 out_seq=0 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.827+0000 7f2abb7fe640 10 client.4434 ms_handle_connect on v2:172.21.15.190:6834/3795516465 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1100082952 <== mds.0 v2:172.21.15.190:6834/3795516465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55face8863e0 con 0x55face9243c0 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2abb7fe640 10 client.4434 handle_command_reply: tid=0 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 client.4434 shutdown 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 -- 172.21.15.135:0/1100082952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 msgr2=0x55face936370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face936370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800cec0 tx=0x7f2ad8004ab0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 -- 172.21.15.135:0/1100082952 shutdown_connections 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55face9243c0 0x55face9247a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 0x55face96c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93b1f0 0x55face96c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2abb7fe640 0 client.4434 ms_handle_reset on v2:172.21.15.190:6834/3795516465 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1100082952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face936370 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 -- 172.21.15.135:0/1100082952 >> 172.21.15.135:0/1100082952 conn(0x55face90e2f0 msgr2=0x55face95c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 -- 172.21.15.135:0/1100082952 shutdown_connections 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 1 -- 172.21.15.135:0/1100082952 wait complete. 2024-04-17T06:09:27.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.828+0000 7f2ae5996740 20 client.4434 trim_cache size 0 max 16384 2024-04-17T06:09:27.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.846+0000 7f2ae5996740 1 Processor -- start 2024-04-17T06:09:27.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.846+0000 7f2ae5996740 1 -- start start 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face9243c0 0x55face923620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face934050 0x55face932810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face932d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae5996740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1ff1c0 con 0x55face934050 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae5996740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0b7bb0 con 0x55face93ae10 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae5996740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc1feb70 con 0x55face9243c0 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face934050 0x55face932810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face934050 0x55face932810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39850/0 (socket says 172.21.15.135:39850) 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 -- 172.21.15.135:0/1356389229 learned_addr learned my addr 172.21.15.135:0/1356389229 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face9243c0 0x55face923620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face932d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 -- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face9243c0 msgr2=0x55face923620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face9243c0 0x55face923620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 -- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 msgr2=0x55face932d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face932d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 -- 172.21.15.135:0/1356389229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55face90dd40 con 0x55face934050 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face932d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face934050 0x55face932810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2acc000c00 tx=0x7f2acc089d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8c139ca48a7413 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ac1ffb640 1 -- 172.21.15.135:0/1356389229 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc076070 con 0x55face934050 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ac1ffb640 1 -- 172.21.15.135:0/1356389229 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc057a60 con 0x55face934050 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.847+0000 7f2ac1ffb640 1 -- 172.21.15.135:0/1356389229 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc01f040 con 0x55face934050 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 -- 172.21.15.135:0/1356389229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face934050 msgr2=0x55face932810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face934050 0x55face932810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2acc000c00 tx=0x7f2acc089d80 comp rx=0 tx=0).stop 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 -- 172.21.15.135:0/1356389229 shutdown_connections 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face93ae10 0x55face932d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face934050 0x55face932810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1356389229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face9243c0 0x55face923620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 -- 172.21.15.135:0/1356389229 >> 172.21.15.135:0/1356389229 conn(0x55face90e2f0 msgr2=0x55face90ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 -- 172.21.15.135:0/1356389229 shutdown_connections 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 -- 172.21.15.135:0/1356389229 wait complete. 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.848+0000 7f2ae5996740 1 -- start start 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.849+0000 7f2ae5996740 1 Processor -- start 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.849+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face9243c0 0x55face9d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.849+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 0x55face941810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.849+0000 7f2ae5996740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face932810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.849+0000 7f2ae5996740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55face93c0c0 con 0x55face93ae10 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.849+0000 7f2ae5996740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55face922390 con 0x55face9243c0 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.849+0000 7f2ae5996740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55face922030 con 0x55face934050 2024-04-17T06:09:27.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae5996740 10 client.0 _ll_register_callbacks cb 0x7f2ad800cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face932810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face932810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39854/0 (socket says 172.21.15.135:39854) 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 -- 172.21.15.135:0/1813258937 learned_addr learned my addr 172.21.15.135:0/1813258937 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ad3fff640 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 0x55face941810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face9243c0 0x55face9d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 -- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 msgr2=0x55face941810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 0x55face941810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 -- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face9243c0 msgr2=0x55face9d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face9243c0 0x55face9d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 -- 172.21.15.135:0/1813258937 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55face9d3010 con 0x55face93ae10 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae1270640 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face932810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009960 tx=0x7f2ad402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8add5fcdec067588 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2abb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:09:27.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1813258937 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4053060 con 0x55face93ae10 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1813258937 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ad4031b80 con 0x55face93ae10 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae5996740 1 -- 172.21.15.135:0/1813258937 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2acc057a60 con 0x55face93ae10 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1813258937 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2ad4035050 con 0x55face93ae10 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae5996740 10 client.4440 fetch_fsmap learned FSMap version 7 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae5996740 1 -- 172.21.15.135:0/1813258937 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55face95c310 con 0x55face93ae10 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1813258937 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc057a60 con 0x55face93ae10 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1813258937 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f2ad4004040 con 0x55face93ae10 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae5996740 10 client.4440 fetch_fsmap finished waiting for FSMap version 7 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae5996740 10 client.4440 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae5996740 20 client.4440 populate_metadata read hostname 'smithi135' 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.850+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55face93b1f0 0x55face9d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 4 client.4440 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 -- 172.21.15.135:0/1813258937 --> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] -- command(tid 0: {"prefix": "status"}) -- 0x55face880250 con 0x55face93b1f0 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55face93b1f0 0x55face9d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae0a6f640 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55face93b1f0 0x55face9d3af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf8ab633598956 server_cookie=84034380d47e9799 in_seq=0 out_seq=0 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2abb7fe640 10 client.4440 ms_handle_connect on v2:172.21.15.190:6834/3795516465 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2abb7fe640 1 -- 172.21.15.135:0/1813258937 <== mds.0 v2:172.21.15.190:6834/3795516465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55face880250 con 0x55face93b1f0 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2abb7fe640 10 client.4440 handle_command_reply: tid=0 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 client.4440 shutdown 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 -- 172.21.15.135:0/1813258937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 msgr2=0x55face932810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face932810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009960 tx=0x7f2ad402e000 comp rx=0 tx=0).stop 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 -- 172.21.15.135:0/1813258937 shutdown_connections 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55face93b1f0 0x55face9d3af0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55face93ae10 0x55face932810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55face934050 0x55face941810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2abb7fe640 0 client.4440 ms_handle_reset on v2:172.21.15.190:6834/3795516465 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 --2- 172.21.15.135:0/1813258937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55face9243c0 0x55face9d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.851+0000 7f2ae5996740 1 -- 172.21.15.135:0/1813258937 >> 172.21.15.135:0/1813258937 conn(0x55face90e2f0 msgr2=0x55face90ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae5996740 1 -- 172.21.15.135:0/1813258937 shutdown_connections 2024-04-17T06:09:27.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae5996740 1 -- 172.21.15.135:0/1813258937 wait complete. 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae5996740 20 client.4440 trim_cache size 0 max 16384 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster_fsid": "96728bf8-42ba-4aa8-88ba-489677bf3f41", 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "whoami": 0, 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "id": 4285, 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "want_state": "up:active", 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "state": "up:active", 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "cephfs", 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "rank_uptime": 7.2197604929999999, 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "mdsmap_epoch": 7, 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch": 24, 2024-04-17T06:09:27.857 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch_barrier": 23, 2024-04-17T06:09:27.858 INFO:teuthology.orchestra.run.smithi135.stdout: "uptime": 14.260370808999999 2024-04-17T06:09:27.858 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-17T06:09:27.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2ac806c8f0 msgr2=0x7f2ac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2ac806c8f0 0x7f2ac806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800d620 tx=0x7f2ad8007810 comp rx=0 tx=0).stop 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc0d0490 msgr2=0x7f2adc15dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc0d0490 0x7f2adc15dc30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00c7c0 tx=0x7f2acc00cc80 comp rx=0 tx=0).stop 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/3793208769 shutdown_connections 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2adc15cc60 0x7f2adc151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2ac806c8f0 0x7f2ac806edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2adc0d0490 0x7f2adc15dc30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 --2- 172.21.15.135:0/3793208769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2adc09af30 0x7f2adc0c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/3793208769 >> 172.21.15.135:0/3793208769 conn(0x7f2adc0b41a0 msgr2=0x7f2adc0b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/3793208769 shutdown_connections 2024-04-17T06:09:27.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.852+0000 7f2ae2cd3640 1 -- 172.21.15.135:0/3793208769 wait complete. 2024-04-17T06:09:27.861 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T06:09:27.861 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-17T06:09:27.861 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T06:09:27.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.967+0000 7f3f07e1b640 1 Processor -- start 2024-04-17T06:09:27.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.968+0000 7f3f07e1b640 1 -- start start 2024-04-17T06:09:27.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.968+0000 7f3f07e1b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0015eaa0 0x7f3f00162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.968+0000 7f3f07e1b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3f000d19d0 0x7f3f000d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.968+0000 7f3f07e1b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f001635e0 con 0x7f3f000ba590 2024-04-17T06:09:27.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.968+0000 7f3f07e1b640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3f00163020 con 0x7f3f000d19d0 2024-04-17T06:09:27.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.968+0000 7f3f07e1b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3f00163300 con 0x7f3f0015eaa0 2024-04-17T06:09:27.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.968+0000 7f3f063b8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3f000d19d0 0x7f3f000d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.969+0000 7f3f063b8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3f000d19d0 0x7f3f000d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39868/0 (socket says 172.21.15.135:39868) 2024-04-17T06:09:27.967 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.969+0000 7f3f063b8640 1 -- 172.21.15.135:0/1961330791 learned_addr learned my addr 172.21.15.135:0/1961330791 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.969+0000 7f3f05bb7640 1 --2- 172.21.15.135:0/1961330791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0015eaa0 0x7f3f00162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.969+0000 7f3f063b8640 1 -- 172.21.15.135:0/1961330791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0015eaa0 msgr2=0x7f3f00162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:27.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.969+0000 7f3f063b8640 1 --2- 172.21.15.135:0/1961330791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0015eaa0 0x7f3f00162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.969+0000 7f3f063b8640 1 -- 172.21.15.135:0/1961330791 >> v1:172.21.15.190:6789/0 conn(0x7f3f000ba590 legacy=0x7f3f000d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.969+0000 7f3f063b8640 1 -- 172.21.15.135:0/1961330791 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f000d2a60 con 0x7f3f000d19d0 2024-04-17T06:09:27.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f063b8640 1 --2- 172.21.15.135:0/1961330791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f000d19d0 0x7f3f000d3260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009870 tx=0x7f3ef802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc6a5a0b82f0bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f04bb5640 1 -- 172.21.15.135:0/1961330791 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef802f840 con 0x7f3f000d19d0 2024-04-17T06:09:27.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f04bb5640 1 -- 172.21.15.135:0/1961330791 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef802f9e0 con 0x7f3f000d19d0 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f04bb5640 1 -- 172.21.15.135:0/1961330791 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef802fdb0 con 0x7f3f000d19d0 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f07e1b640 1 -- 172.21.15.135:0/1961330791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f000d19d0 msgr2=0x7f3f000d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/1961330791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f000d19d0 0x7f3f000d3260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009870 tx=0x7f3ef802ba20 comp rx=0 tx=0).stop 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f07e1b640 1 -- 172.21.15.135:0/1961330791 shutdown_connections 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/1961330791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f000d19d0 0x7f3f000d3260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/1961330791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f0015eaa0 0x7f3f00162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f07e1b640 1 -- 172.21.15.135:0/1961330791 >> 172.21.15.135:0/1961330791 conn(0x7f3f0006ce10 msgr2=0x7f3f0006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f07e1b640 1 -- 172.21.15.135:0/1961330791 shutdown_connections 2024-04-17T06:09:27.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.970+0000 7f3f07e1b640 1 -- 172.21.15.135:0/1961330791 wait complete. 2024-04-17T06:09:27.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.971+0000 7f3f07e1b640 1 Processor -- start 2024-04-17T06:09:27.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.971+0000 7f3f07e1b640 1 -- start start 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f07e1b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000ba590 0x7f3f00156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f07e1b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f000d19d0 0x7f3f001611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f07e1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f0015eaa0 0x7f3f00161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f07e1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00162de0 con 0x7f3f0015eaa0 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f07e1b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00162c60 con 0x7f3f000ba590 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f07e1b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0015ee80 con 0x7f3f000d19d0 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f063b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f0015eaa0 0x7f3f00161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f063b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f0015eaa0 0x7f3f00161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39870/0 (socket says 172.21.15.135:39870) 2024-04-17T06:09:27.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f063b8640 1 -- 172.21.15.135:0/3717213550 learned_addr learned my addr 172.21.15.135:0/3717213550 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f053b6640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f000d19d0 0x7f3f001611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.972+0000 7f3f05bb7640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000ba590 0x7f3f00156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3f063b8640 1 -- 172.21.15.135:0/3717213550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f000d19d0 msgr2=0x7f3f001611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3f063b8640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f000d19d0 0x7f3f001611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3f063b8640 1 -- 172.21.15.135:0/3717213550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000ba590 msgr2=0x7f3f00156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3f063b8640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000ba590 0x7f3f00156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3f063b8640 1 -- 172.21.15.135:0/3717213550 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f001f9670 con 0x7f3f0015eaa0 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3f063b8640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f0015eaa0 0x7f3f00161720 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009870 tx=0x7f3ef8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3efe7fc640 1 -- 172.21.15.135:0/3717213550 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8037c40 con 0x7f3f0015eaa0 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3efe7fc640 1 -- 172.21.15.135:0/3717213550 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef8046070 con 0x7f3f0015eaa0 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3efe7fc640 1 -- 172.21.15.135:0/3717213550 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8041400 con 0x7f3f0015eaa0 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f000cba20 con 0x7f3f0015eaa0 2024-04-17T06:09:27.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.973+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f000cbfe0 con 0x7f3f0015eaa0 2024-04-17T06:09:27.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.974+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f00068a20 con 0x7f3f0015eaa0 2024-04-17T06:09:27.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.974+0000 7f3efe7fc640 1 -- 172.21.15.135:0/3717213550 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ef8040070 con 0x7f3f0015eaa0 2024-04-17T06:09:27.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.975+0000 7f3efe7fc640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3eec06c920 0x7f3eec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:27.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.975+0000 7f3f05bb7640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3eec06c920 0x7f3eec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:27.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.975+0000 7f3efe7fc640 1 -- 172.21.15.135:0/3717213550 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3ef80ae370 con 0x7f3f0015eaa0 2024-04-17T06:09:27.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.975+0000 7f3f05bb7640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3eec06c920 0x7f3eec06ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0011340 tx=0x7f3ef000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:27.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:27.979+0000 7f3efe7fc640 1 -- 172.21.15.135:0/3717213550 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ef8033030 con 0x7f3f0015eaa0 2024-04-17T06:09:28.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.192+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f000d24e0 con 0x7f3f0015eaa0 2024-04-17T06:09:28.190 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.192+0000 7f3efe7fc640 1 -- 172.21.15.135:0/3717213550 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f3ef807bed0 con 0x7f3f0015eaa0 2024-04-17T06:09:28.190 INFO:teuthology.orchestra.run.smithi135.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3eec06c920 msgr2=0x7f3eec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3eec06c920 0x7f3eec06ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0011340 tx=0x7f3ef000b040 comp rx=0 tx=0).stop 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f0015eaa0 msgr2=0x7f3f00161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f0015eaa0 0x7f3f00161720 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009870 tx=0x7f3ef8004730 comp rx=0 tx=0).stop 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 shutdown_connections 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3eec06c920 0x7f3eec06ede0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f0015eaa0 0x7f3f00161720 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f000d19d0 0x7f3f001611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 --2- 172.21.15.135:0/3717213550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f000ba590 0x7f3f00156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 >> 172.21.15.135:0/3717213550 conn(0x7f3f0006ce10 msgr2=0x7f3f000b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:28.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.194+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 shutdown_connections 2024-04-17T06:09:28.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28.195+0000 7f3f07e1b640 1 -- 172.21.15.135:0/3717213550 wait complete. 2024-04-17T06:09:28.203 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-17T06:09:28.203 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T06:09:28.204 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2024-04-17T06:09:28.204 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-17T06:09:28.204 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-17T06:09:28.204 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-17T06:09:28.204 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T06:09:28.204 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T06:09:28.204 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-17T06:09:28.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:28.204 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2024-04-17T06:09:28.219 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f0 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f1 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:28.220 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:28.221 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T06:09:28.221 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:28.221 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T06:09:28.221 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42753sec preferred_lft 42753sec 2024-04-17T06:09:28.221 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::ec4:7aff:fe88:8083/64 scope link 2024-04-17T06:09:28.221 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:28.221 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T06:09:28.222 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:28.222 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:28.222 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add name ceph-brx type bridge 2024-04-17T06:09:28.222 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr flush dev ceph-brx 2024-04-17T06:09:28.222 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set ceph-brx up 2024-04-17T06:09:28.222 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T06:09:28.222 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:28.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:28.484 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:28.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:28.490 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T06:09:28.519 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-04-17T06:09:28.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:28.521 DEBUG:teuthology.orchestra.run.smithi135:> route 2024-04-17T06:09:28.590 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2024-04-17T06:09:28.591 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T06:09:28.591 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T06:09:28.591 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T06:09:28.591 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T06:09:28.591 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:28.592 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:28.592 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T06:09:28.592 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T06:09:28.592 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T06:09:28.592 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:28.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:29.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:29.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:29.296 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-17T06:09:29.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:29.311 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list-id 2024-04-17T06:09:29.367 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:29.367 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:29.367 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T06:09:29.367 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-17T06:09:29.367 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:29.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:29.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:29.492 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-17T06:09:29.493 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:29.493 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:29.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-17T06:09:29.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T06:09:29.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-17T06:09:29.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-17T06:09:29.493 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-17T06:09:29.493 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:29.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:29.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:29.634 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:29.634 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:29.634 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.0 up 2024-04-17T06:09:29.634 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T06:09:29.634 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:29.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:29.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:29.719 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-17T06:09:29.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:29.720 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-17T06:09:29.735 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-17T06:09:29.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:29.735 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-17T06:09:29.791 DEBUG:teuthology.orchestra.run.smithi135:> sudo modprobe fuse 2024-04-17T06:09:29.861 DEBUG:teuthology.orchestra.run.smithi135:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T06:09:29.917 INFO:teuthology.orchestra.run.smithi135.stdout:/proc 2024-04-17T06:09:29.917 INFO:teuthology.orchestra.run.smithi135.stdout:/sys 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/dev 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/kernel/security 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/dev/shm 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/dev/pts 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/run 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/cgroup 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/pstore 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/bpf 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/ 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/selinux 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/dev/mqueue 2024-04-17T06:09:29.918 INFO:teuthology.orchestra.run.smithi135.stdout:/dev/hugepages 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/kernel/debug 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/kernel/tracing 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/fuse/connections 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/kernel/config 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/run/user/1000 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T06:09:29.919 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T06:09:29.920 INFO:teuthology.orchestra.run.smithi135.stdout:/run/netns 2024-04-17T06:09:29.920 INFO:teuthology.orchestra.run.smithi135.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T06:09:29.920 INFO:teuthology.orchestra.run.smithi135.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T06:09:29.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:29.920 DEBUG:teuthology.orchestra.run.smithi135:> ls /sys/fs/fuse/connections 2024-04-17T06:09:29.973 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-17T06:09:29.974 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-17T06:09:30.016 DEBUG:teuthology.orchestra.run.smithi135:> sudo modprobe fuse 2024-04-17T06:09:30.045 DEBUG:teuthology.orchestra.run.smithi135:> cat /proc/self/mounts | awk '{print $2}' 2024-04-17T06:09:30.102 INFO:teuthology.orchestra.run.smithi135.stdout:/proc 2024-04-17T06:09:30.102 INFO:teuthology.orchestra.run.smithi135.stdout:/sys 2024-04-17T06:09:30.102 INFO:teuthology.orchestra.run.smithi135.stdout:/dev 2024-04-17T06:09:30.102 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/kernel/security 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/dev/shm 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/dev/pts 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/run 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/cgroup 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/pstore 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/bpf 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/ 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/selinux 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/proc/sys/fs/binfmt_misc 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/dev/mqueue 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/dev/hugepages 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/kernel/debug 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/kernel/tracing 2024-04-17T06:09:30.103 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/fs/fuse/connections 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/sys/kernel/config 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/run/credentials/systemd-sysctl.service 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/nfs/rpc_pipefs 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/run/user/1000 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph/osd/ceph-0 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph/osd/ceph-1 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph/osd/ceph-2 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/var/lib/ceph/osd/ceph-3 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/run/netns 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T06:09:30.104 INFO:teuthology.orchestra.run.smithi135.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T06:09:30.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:30.105 DEBUG:teuthology.orchestra.run.smithi135:> ls /sys/fs/fuse/connections 2024-04-17T06:09:30.106 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi135.stderr:2024-04-17T06:09:30.108+0000 7f7bad236480 -1 init, newargv = 0x55b51a69e620 newargc=15 2024-04-17T06:09:30.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi135.stderr:ceph-fuse[44587]: starting ceph client 2024-04-17T06:09:30.113 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi135.stderr:ceph-fuse[44587]: starting fuse 2024-04-17T06:09:30.122 INFO:teuthology.orchestra.run.smithi135.stdout:41 2024-04-17T06:09:30.122 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-17T06:09:30.122 DEBUG:teuthology.orchestra.run.smithi135:> sudo stdin-killer -- python3 -c ' 2024-04-17T06:09:30.122 DEBUG:teuthology.orchestra.run.smithi135:> import glob 2024-04-17T06:09:30.122 DEBUG:teuthology.orchestra.run.smithi135:> import re 2024-04-17T06:09:30.122 DEBUG:teuthology.orchestra.run.smithi135:> import os 2024-04-17T06:09:30.122 DEBUG:teuthology.orchestra.run.smithi135:> import subprocess 2024-04-17T06:09:30.122 DEBUG:teuthology.orchestra.run.smithi135:> 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> def _find_admin_socket(client_name): 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> files = glob.glob(asok_path) 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> # Given a non-glob path, it better be there 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> if "*" not in asok_path: 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> assert(len(files) == 1) 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> return files[0] 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> for f in files: 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> contents = proc_f.read() 2024-04-17T06:09:30.123 DEBUG:teuthology.orchestra.run.smithi135:> if mountpoint in contents: 2024-04-17T06:09:30.124 DEBUG:teuthology.orchestra.run.smithi135:> return f 2024-04-17T06:09:30.124 DEBUG:teuthology.orchestra.run.smithi135:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-17T06:09:30.124 DEBUG:teuthology.orchestra.run.smithi135:> 2024-04-17T06:09:30.124 DEBUG:teuthology.orchestra.run.smithi135:> print(_find_admin_socket("client.admin")) 2024-04-17T06:09:30.124 DEBUG:teuthology.orchestra.run.smithi135:> ' 2024-04-17T06:09:30.244 INFO:teuthology.orchestra.run.smithi135.stdout:/var/run/ceph/ceph-client.admin.44587.asok 2024-04-17T06:09:30.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:30.255 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44587.asok 2024-04-17T06:09:30.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:30.255 DEBUG:teuthology.orchestra.run.smithi135:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44587.asok status 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: "metadata": { 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_sha1": "61531d579f13bd3cc7ef7522484ee03eec1daad6", 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_version": "ceph version 19.0.0-2862-g61531d57 (61531d579f13bd3cc7ef7522484ee03eec1daad6) squid (dev)", 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: "entity_id": "admin", 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: "hostname": "smithi135", 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: "pid": "44587", 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: "root": "/" 2024-04-17T06:09:30.393 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_count": 0, 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_pinned_count": 0, 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "id": 4452, 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "inst": { 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "name": { 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "type": "client", 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "num": 4452 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "addr": { 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "type": "v1", 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "addr": "192.168.0.1:0", 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: "nonce": 1612355662 2024-04-17T06:09:30.394 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "addr": { 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "type": "v1", 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "addr": "192.168.0.1:0", 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "nonce": 1612355662 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "inst_str": "client.4452 192.168.0.1:0/1612355662", 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "addr_str": "192.168.0.1:0/1612355662", 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "inode_count": 1, 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_epoch": 7, 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_epoch": 24, 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_epoch_barrier": 0, 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "blocklisted": false, 2024-04-17T06:09:30.395 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "cephfs" 2024-04-17T06:09:30.396 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-17T06:09:30.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:30.401 DEBUG:teuthology.orchestra.run.smithi135:> stat --file-system '--printf=%T 2024-04-17T06:09:30.401 DEBUG:teuthology.orchestra.run.smithi135:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-17T06:09:30.417 INFO:teuthology.orchestra.run.smithi135.stdout:fuseblk 2024-04-17T06:09:30.417 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-17T06:09:30.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:30.417 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-17T06:09:30.487 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-17T06:09:30.579 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:30.596 INFO:teuthology.orchestra.run.smithi135.stdout:# file: . 2024-04-17T06:09:30.596 INFO:teuthology.orchestra.run.smithi135.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-17T06:09:30.596 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:30.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:30.602 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:09:30.602 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:09:30.623 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:30.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.768+0000 7f7326b3d640 1 Processor -- start 2024-04-17T06:09:30.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.769+0000 7f7326b3d640 1 -- start start 2024-04-17T06:09:30.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.769+0000 7f7326b3d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f732015bb00 0x7f73200b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.769+0000 7f7326b3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f73200ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.769+0000 7f7326b3d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73200b3a50 con 0x7f73200b3590 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.769+0000 7f7326b3d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f73200b4660 con 0x7f732015bb00 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.769+0000 7f7326b3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f73200b3d30 con 0x7f73200afdd0 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.769+0000 7f73248d9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f732015bb00 0x7f73200b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f73248d9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f732015bb00 0x7f73200b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39874/0 (socket says 172.21.15.135:39874) 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f731ffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f73200b3590 0x7f73200af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35826/0 (socket says 172.21.15.135:35826) 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f73248d9640 1 -- 172.21.15.135:0/1792959159 learned_addr learned my addr 172.21.15.135:0/1792959159 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f73250da640 1 --2- 172.21.15.135:0/1792959159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f73200ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f73248d9640 1 -- 172.21.15.135:0/1792959159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 msgr2=0x7f73200ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f73248d9640 1 --2- 172.21.15.135:0/1792959159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f73200ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f73248d9640 1 -- 172.21.15.135:0/1792959159 >> v1:172.21.15.135:6789/0 conn(0x7f73200b3590 legacy=0x7f73200af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f73248d9640 1 -- 172.21.15.135:0/1792959159 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200b4010 con 0x7f732015bb00 2024-04-17T06:09:30.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.770+0000 7f73250da640 1 --2- 172.21.15.135:0/1792959159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f73200ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:30.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f73248d9640 1 --2- 172.21.15.135:0/1792959159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f73200b5150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7310009870 tx=0x7f731002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca0598cd4307c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:30.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f731f7fe640 1 -- 172.21.15.135:0/1792959159 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310037070 con 0x7f732015bb00 2024-04-17T06:09:30.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f731f7fe640 1 -- 172.21.15.135:0/1792959159 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7310004230 con 0x7f732015bb00 2024-04-17T06:09:30.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f731f7fe640 1 -- 172.21.15.135:0/1792959159 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310003610 con 0x7f732015bb00 2024-04-17T06:09:30.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f7326b3d640 1 -- 172.21.15.135:0/1792959159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 msgr2=0x7f73200b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:30.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f7326b3d640 1 --2- 172.21.15.135:0/1792959159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f73200b5150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7310009870 tx=0x7f731002b7e0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f7326b3d640 1 -- 172.21.15.135:0/1792959159 shutdown_connections 2024-04-17T06:09:30.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f7326b3d640 1 --2- 172.21.15.135:0/1792959159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f73200ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f7326b3d640 1 --2- 172.21.15.135:0/1792959159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f73200b5150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f7326b3d640 1 -- 172.21.15.135:0/1792959159 >> 172.21.15.135:0/1792959159 conn(0x7f73200d0520 msgr2=0x7f73200d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:30.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f7326b3d640 1 -- 172.21.15.135:0/1792959159 shutdown_connections 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.771+0000 7f7326b3d640 1 -- 172.21.15.135:0/1792959159 wait complete. 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.772+0000 7f7326b3d640 1 Processor -- start 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.772+0000 7f7326b3d640 1 -- start start 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f7326b3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f732014ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f7326b3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200b3590 0x7f732015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f7326b3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f732015e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f7326b3d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73200b5810 con 0x7f732015bb00 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f7326b3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f73200b5690 con 0x7f73200b3590 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f7326b3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f73200cec60 con 0x7f73200afdd0 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f73250da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f732015e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f73250da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f732015e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39876/0 (socket says 172.21.15.135:39876) 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.773+0000 7f73250da640 1 -- 172.21.15.135:0/3189225565 learned_addr learned my addr 172.21.15.135:0/3189225565 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f73248d9640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f732014ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f731ffff640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200b3590 0x7f732015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:30.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f73250da640 1 -- 172.21.15.135:0/3189225565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 msgr2=0x7f732014ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f73250da640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f732014ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f73250da640 1 -- 172.21.15.135:0/3189225565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200b3590 msgr2=0x7f732015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f73250da640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200b3590 0x7f732015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f73250da640 1 -- 172.21.15.135:0/3189225565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200ad860 con 0x7f732015bb00 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f73250da640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f732015e6b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f731800cef0 tx=0x7f731800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f731d7fa640 1 -- 172.21.15.135:0/3189225565 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731801a070 con 0x7f732015bb00 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f731d7fa640 1 -- 172.21.15.135:0/3189225565 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731800d860 con 0x7f732015bb00 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73200adb00 con 0x7f732015bb00 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f731d7fa640 1 -- 172.21.15.135:0/3189225565 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731800dc50 con 0x7f732015bb00 2024-04-17T06:09:30.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.774+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73200ae0c0 con 0x7f732015bb00 2024-04-17T06:09:30.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.775+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7320074940 con 0x7f732015bb00 2024-04-17T06:09:30.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.776+0000 7f731d7fa640 1 -- 172.21.15.135:0/3189225565 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f731801d070 con 0x7f732015bb00 2024-04-17T06:09:30.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.776+0000 7f731d7fa640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:30.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.776+0000 7f731d7fa640 1 -- 172.21.15.135:0/3189225565 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f731808c740 con 0x7f732015bb00 2024-04-17T06:09:30.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.776+0000 7f73248d9640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:30.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.777+0000 7f73248d9640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7310034000 tx=0x7f7310008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:30.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.780+0000 7f731d7fa640 1 -- 172.21.15.135:0/3189225565 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f731805a210 con 0x7f732015bb00 2024-04-17T06:09:30.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.960+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73201576c0 con 0x7f732015bb00 2024-04-17T06:09:30.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.961+0000 7f731d7fa640 1 -- 172.21.15.135:0/3189225565 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f731805e510 con 0x7f732015bb00 2024-04-17T06:09:30.959 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:30.959 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 msgr2=0x7f730006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7310034000 tx=0x7f7310008030 comp rx=0 tx=0).stop 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 msgr2=0x7f732015e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f732015e6b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f731800cef0 tx=0x7f731800a710 comp rx=0 tx=0).stop 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 shutdown_connections 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f732015bb00 0x7f732015e6b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73200b3590 0x7f732015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 --2- 172.21.15.135:0/3189225565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200afdd0 0x7f732014ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 >> 172.21.15.135:0/3189225565 conn(0x7f73200d0520 msgr2=0x7f7320157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 shutdown_connections 2024-04-17T06:09:30.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:30.963+0000 7f7326b3d640 1 -- 172.21.15.135:0/3189225565 wait complete. 2024-04-17T06:09:30.972 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:09:30.972 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:31.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.080+0000 7ffabef59640 1 Processor -- start 2024-04-17T06:09:31.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.080+0000 7ffabef59640 1 -- start start 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffabef59640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffab8073a40 0x7ffab815d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffabef59640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab80ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffabef59640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab80cec60 con 0x7ffab80d0900 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffabef59640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ffab80cef40 con 0x7ffab8073a40 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffabef59640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffab815d000 con 0x7ffab815cae0 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffab7fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffab8073a40 0x7ffab815d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffabd4f6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab80ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffabccf5640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ffab80d0900 0x7ffab80d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:58440/0 (socket says 172.21.15.135:58440) 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.081+0000 7ffabccf5640 1 -- 172.21.15.135:0/3571358785 learned_addr learned my addr 172.21.15.135:0/3571358785 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.082+0000 7ffab77fe640 1 -- 172.21.15.135:0/3571358785 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2197303494 0 0) 0x7ffab80cec60 con 0x7ffab80d0900 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.082+0000 7ffab77fe640 1 -- 172.21.15.135:0/3571358785 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaa4003520 con 0x7ffab80d0900 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.082+0000 7ffabd4f6640 1 -- 172.21.15.135:0/3571358785 >> v2:172.21.15.190:3300/0 conn(0x7ffab8073a40 msgr2=0x7ffab815d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.082+0000 7ffabd4f6640 1 --2- 172.21.15.135:0/3571358785 >> v2:172.21.15.190:3300/0 conn(0x7ffab8073a40 0x7ffab815d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.082+0000 7ffabd4f6640 1 -- 172.21.15.135:0/3571358785 >> v1:172.21.15.190:6789/0 conn(0x7ffab80d0900 legacy=0x7ffab80d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.082+0000 7ffabd4f6640 1 -- 172.21.15.135:0/3571358785 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab815d2e0 con 0x7ffab815cae0 2024-04-17T06:09:31.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.082+0000 7ffab7fff640 1 --2- 172.21.15.135:0/3571358785 >> v2:172.21.15.190:3300/0 conn(0x7ffab8073a40 0x7ffab815d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:31.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.082+0000 7ffabd4f6640 1 --2- 172.21.15.135:0/3571358785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab80ce2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009d90 tx=0x7ffab002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac21a162f7510d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffab77fe640 1 -- 172.21.15.135:0/3571358785 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0039070 con 0x7ffab815cae0 2024-04-17T06:09:31.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffab77fe640 1 -- 172.21.15.135:0/3571358785 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab002cc50 con 0x7ffab815cae0 2024-04-17T06:09:31.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffab77fe640 1 -- 172.21.15.135:0/3571358785 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0034440 con 0x7ffab815cae0 2024-04-17T06:09:31.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffabef59640 1 -- 172.21.15.135:0/3571358785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 msgr2=0x7ffab80ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.082 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffabef59640 1 --2- 172.21.15.135:0/3571358785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab80ce2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009d90 tx=0x7ffab002bea0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffabef59640 1 -- 172.21.15.135:0/3571358785 shutdown_connections 2024-04-17T06:09:31.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffabef59640 1 --2- 172.21.15.135:0/3571358785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab80ce2f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffabef59640 1 --2- 172.21.15.135:0/3571358785 >> v2:172.21.15.190:3300/0 conn(0x7ffab8073a40 0x7ffab815d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffabef59640 1 -- 172.21.15.135:0/3571358785 >> 172.21.15.135:0/3571358785 conn(0x7ffab80d15e0 msgr2=0x7ffab80d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:31.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffabef59640 1 -- 172.21.15.135:0/3571358785 shutdown_connections 2024-04-17T06:09:31.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.083+0000 7ffabef59640 1 -- 172.21.15.135:0/3571358785 wait complete. 2024-04-17T06:09:31.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.084+0000 7ffabef59640 1 Processor -- start 2024-04-17T06:09:31.083 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.084+0000 7ffabef59640 1 -- start start 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.085+0000 7ffabef59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffab8073a40 0x7ffab81f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.085+0000 7ffabef59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab80d0900 0x7ffab815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.085+0000 7ffabef59640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab81f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.085+0000 7ffabef59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffab80ce9b0 con 0x7ffab8073a40 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.085+0000 7ffabef59640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffab80ce830 con 0x7ffab80d0900 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.085+0000 7ffabef59640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffab8160450 con 0x7ffab815cae0 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.085+0000 7ffabccf5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffab8073a40 0x7ffab81f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabccf5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffab8073a40 0x7ffab81f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39884/0 (socket says 172.21.15.135:39884) 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabccf5640 1 -- 172.21.15.135:0/267898900 learned_addr learned my addr 172.21.15.135:0/267898900 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabd4f6640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab81f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffab7fff640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab80d0900 0x7ffab815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabccf5640 1 -- 172.21.15.135:0/267898900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 msgr2=0x7ffab81f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabccf5640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab81f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabccf5640 1 -- 172.21.15.135:0/267898900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab80d0900 msgr2=0x7ffab815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabccf5640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab80d0900 0x7ffab815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabccf5640 1 -- 172.21.15.135:0/267898900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab81f6be0 con 0x7ffab8073a40 2024-04-17T06:09:31.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabccf5640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffab8073a40 0x7ffab81f66a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800c4d0 tx=0x7ffaa800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffab57fa640 1 -- 172.21.15.135:0/267898900 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8017070 con 0x7ffab8073a40 2024-04-17T06:09:31.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffab57fa640 1 -- 172.21.15.135:0/267898900 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa8005d00 con 0x7ffab8073a40 2024-04-17T06:09:31.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab81f30d0 con 0x7ffab8073a40 2024-04-17T06:09:31.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffab57fa640 1 -- 172.21.15.135:0/267898900 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80072b0 con 0x7ffab8073a40 2024-04-17T06:09:31.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.086+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab81f5ea0 con 0x7ffab8073a40 2024-04-17T06:09:31.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.087+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab8074940 con 0x7ffab8073a40 2024-04-17T06:09:31.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.088+0000 7ffab57fa640 1 -- 172.21.15.135:0/267898900 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffaa801a070 con 0x7ffab8073a40 2024-04-17T06:09:31.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.088+0000 7ffab57fa640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffa9806c8f0 0x7ffa9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.088+0000 7ffab57fa640 1 -- 172.21.15.135:0/267898900 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ffaa808b140 con 0x7ffab8073a40 2024-04-17T06:09:31.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.088+0000 7ffab7fff640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffa9806c8f0 0x7ffa9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.089+0000 7ffab7fff640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffa9806c8f0 0x7ffa9806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009560 tx=0x7ffaa0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.093+0000 7ffab57fa640 1 -- 172.21.15.135:0/267898900 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffab8074940 con 0x7ffab8073a40 2024-04-17T06:09:31.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.266+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffab8068a20 con 0x7ffab8073a40 2024-04-17T06:09:31.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.267+0000 7ffab57fa640 1 -- 172.21.15.135:0/267898900 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7ffab81f6be0 con 0x7ffab8073a40 2024-04-17T06:09:31.265 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffa9806c8f0 msgr2=0x7ffa9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffa9806c8f0 0x7ffa9806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009560 tx=0x7ffaa0009190 comp rx=0 tx=0).stop 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffab8073a40 msgr2=0x7ffab81f66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffab8073a40 0x7ffab81f66a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800c4d0 tx=0x7ffaa800c990 comp rx=0 tx=0).stop 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 shutdown_connections 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffab815cae0 0x7ffab81f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffab80d0900 0x7ffab815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffa9806c8f0 0x7ffa9806edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 --2- 172.21.15.135:0/267898900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffab8073a40 0x7ffab81f66a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 >> 172.21.15.135:0/267898900 conn(0x7ffab80d15e0 msgr2=0x7ffab80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 shutdown_connections 2024-04-17T06:09:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.269+0000 7ffabef59640 1 -- 172.21.15.135:0/267898900 wait complete. 2024-04-17T06:09:31.278 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:09:31.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.386+0000 7f152ecd3640 1 Processor -- start 2024-04-17T06:09:31.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.386+0000 7f152ecd3640 1 -- start start 2024-04-17T06:09:31.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f152ecd3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f152815ec70 0x7f152806b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f152ecd3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f15280d0630 0x7f15280cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f152ecd3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1528160060 con 0x7f15280cfc40 2024-04-17T06:09:31.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f152ecd3640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f15280b60a0 con 0x7f15280d0630 2024-04-17T06:09:31.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f152ecd3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f15280b5ae0 con 0x7f152815ec70 2024-04-17T06:09:31.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f1527fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f15280d0630 0x7f15280cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f1527fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f15280d0630 0x7f15280cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39886/0 (socket says 172.21.15.135:39886) 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f1527fff640 1 -- 172.21.15.135:0/1716869796 learned_addr learned my addr 172.21.15.135:0/1716869796 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f152ca6f640 1 --2- 172.21.15.135:0/1716869796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f152815ec70 0x7f152806b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f1527fff640 1 -- 172.21.15.135:0/1716869796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f152815ec70 msgr2=0x7f152806b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f1527fff640 1 --2- 172.21.15.135:0/1716869796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f152815ec70 0x7f152806b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f1527fff640 1 -- 172.21.15.135:0/1716869796 >> v1:172.21.15.190:6789/0 conn(0x7f15280cfc40 legacy=0x7f1528162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.387+0000 7f1527fff640 1 -- 172.21.15.135:0/1716869796 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15280b5dc0 con 0x7f15280d0630 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.388+0000 7f1527fff640 1 --2- 172.21.15.135:0/1716869796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f15280cf700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f151c009870 tx=0x7f151c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9502b87e77347475 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.388+0000 7f15277fe640 1 -- 172.21.15.135:0/1716869796 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c035020 con 0x7f15280d0630 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.388+0000 7f15277fe640 1 -- 172.21.15.135:0/1716869796 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151c031640 con 0x7f15280d0630 2024-04-17T06:09:31.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.388+0000 7f15277fe640 1 -- 172.21.15.135:0/1716869796 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c02d040 con 0x7f15280d0630 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.389+0000 7f152ecd3640 1 -- 172.21.15.135:0/1716869796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 msgr2=0x7f15280cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.389+0000 7f152ecd3640 1 --2- 172.21.15.135:0/1716869796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f15280cf700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f151c009870 tx=0x7f151c02b8f0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.389+0000 7f152ecd3640 1 -- 172.21.15.135:0/1716869796 shutdown_connections 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.389+0000 7f152ecd3640 1 --2- 172.21.15.135:0/1716869796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f15280cf700 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.389+0000 7f152ecd3640 1 --2- 172.21.15.135:0/1716869796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f152815ec70 0x7f152806b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.389+0000 7f152ecd3640 1 -- 172.21.15.135:0/1716869796 >> 172.21.15.135:0/1716869796 conn(0x7f15280cd7e0 msgr2=0x7f15280cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.389+0000 7f152ecd3640 1 -- 172.21.15.135:0/1716869796 shutdown_connections 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.389+0000 7f152ecd3640 1 -- 172.21.15.135:0/1716869796 wait complete. 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.390+0000 7f152ecd3640 1 Processor -- start 2024-04-17T06:09:31.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.390+0000 7f152ecd3640 1 -- start start 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f152ecd3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cfc40 0x7f15281618d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f152ecd3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f152815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f152ecd3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f152815ec70 0x7f152815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f152ecd3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15280d0ab0 con 0x7f15280d0630 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f152ecd3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f15280b4160 con 0x7f152815ec70 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f152ecd3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f15280d00d0 con 0x7f15280cfc40 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f1527fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f152815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f1527fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f152815c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39894/0 (socket says 172.21.15.135:39894) 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f1527fff640 1 -- 172.21.15.135:0/2232139903 learned_addr learned my addr 172.21.15.135:0/2232139903 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f152ca6f640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cfc40 0x7f15281618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f152d270640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f152815ec70 0x7f152815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f1527fff640 1 -- 172.21.15.135:0/2232139903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cfc40 msgr2=0x7f15281618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f1527fff640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cfc40 0x7f15281618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f1527fff640 1 -- 172.21.15.135:0/2232139903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f152815ec70 msgr2=0x7f152815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f1527fff640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f152815ec70 0x7f152815c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.391+0000 7f1527fff640 1 -- 172.21.15.135:0/2232139903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15281495b0 con 0x7f15280d0630 2024-04-17T06:09:31.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.392+0000 7f1527fff640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f152815c010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f151c02b930 tx=0x7f151c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.392+0000 7f15257fa640 1 -- 172.21.15.135:0/2232139903 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c040070 con 0x7f15280d0630 2024-04-17T06:09:31.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.392+0000 7f15257fa640 1 -- 172.21.15.135:0/2232139903 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151c03e070 con 0x7f15280d0630 2024-04-17T06:09:31.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.392+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1528146760 con 0x7f15280d0630 2024-04-17T06:09:31.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.392+0000 7f15257fa640 1 -- 172.21.15.135:0/2232139903 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c035040 con 0x7f15280d0630 2024-04-17T06:09:31.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.392+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1528146d20 con 0x7f15280d0630 2024-04-17T06:09:31.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.393+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1528074940 con 0x7f15280d0630 2024-04-17T06:09:31.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.393+0000 7f15257fa640 1 -- 172.21.15.135:0/2232139903 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f151c038020 con 0x7f15280d0630 2024-04-17T06:09:31.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.394+0000 7f15257fa640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f151406c8f0 0x7f151406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.394+0000 7f15257fa640 1 -- 172.21.15.135:0/2232139903 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f151c0ad8b0 con 0x7f15280d0630 2024-04-17T06:09:31.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.394+0000 7f152ca6f640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f151406c8f0 0x7f151406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.394+0000 7f152ca6f640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f151406c8f0 0x7f151406edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1518006fd0 tx=0x7f1518008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.398+0000 7f15257fa640 1 -- 172.21.15.135:0/2232139903 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f151c07b9a0 con 0x7f15280d0630 2024-04-17T06:09:31.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.571+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f152806d740 con 0x7f15280d0630 2024-04-17T06:09:31.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.572+0000 7f15257fa640 1 -- 172.21.15.135:0/2232139903 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f151c07b2a0 con 0x7f15280d0630 2024-04-17T06:09:31.570 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f151406c8f0 msgr2=0x7f151406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f151406c8f0 0x7f151406edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1518006fd0 tx=0x7f1518008040 comp rx=0 tx=0).stop 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 msgr2=0x7f152815c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f152815c010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f151c02b930 tx=0x7f151c0046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 shutdown_connections 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f152815ec70 0x7f152815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f151406c8f0 0x7f151406edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15280d0630 0x7f152815c010 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.572 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 --2- 172.21.15.135:0/2232139903 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15280cfc40 0x7f15281618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 >> 172.21.15.135:0/2232139903 conn(0x7f15280cd7e0 msgr2=0x7f152815f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:31.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 shutdown_connections 2024-04-17T06:09:31.573 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.574+0000 7f152ecd3640 1 -- 172.21.15.135:0/2232139903 wait complete. 2024-04-17T06:09:31.583 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:09:31.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.691+0000 7fb733b9d640 1 Processor -- start 2024-04-17T06:09:31.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.691+0000 7fb733b9d640 1 -- start start 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb733b9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15f800 0x7fb72c070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb733b9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb72c070c90 0x7fb72c0d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb733b9d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb72c06bb30 con 0x7fb72c15fbe0 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb733b9d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb72c06c8f0 con 0x7fb72c070c90 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb733b9d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c06f070 con 0x7fb72c15f800 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb731138640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb72c070c90 0x7fb72c0d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb731939640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15f800 0x7fb72c070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb73213a640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fb72c15fbe0 0x7fb72c162530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:58456/0 (socket says 172.21.15.135:58456) 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.692+0000 7fb73213a640 1 -- 172.21.15.135:0/1553744236 learned_addr learned my addr 172.21.15.135:0/1553744236 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.693+0000 7fb731138640 1 -- 172.21.15.135:0/1553744236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15f800 msgr2=0x7fb72c070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.693+0000 7fb731138640 1 --2- 172.21.15.135:0/1553744236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15f800 0x7fb72c070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.691 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.693+0000 7fb731138640 1 -- 172.21.15.135:0/1553744236 >> v1:172.21.15.190:6789/0 conn(0x7fb72c15fbe0 legacy=0x7fb72c162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.693+0000 7fb730937640 1 -- 172.21.15.135:0/1553744236 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 958676528 0 0) 0x7fb72c06bb30 con 0x7fb72c15fbe0 2024-04-17T06:09:31.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.693+0000 7fb731138640 1 -- 172.21.15.135:0/1553744236 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c06be10 con 0x7fb72c070c90 2024-04-17T06:09:31.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.693+0000 7fb731939640 1 --2- 172.21.15.135:0/1553744236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15f800 0x7fb72c070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb731138640 1 --2- 172.21.15.135:0/1553744236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c070c90 0x7fb72c0d0810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0097d0 tx=0x7fb71c02b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c2f3c04f15f9fa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb730937640 1 -- 172.21.15.135:0/1553744236 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c035020 con 0x7fb72c070c90 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb730937640 1 -- 172.21.15.135:0/1553744236 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c031890 con 0x7fb72c070c90 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb730937640 1 -- 172.21.15.135:0/1553744236 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c031c60 con 0x7fb72c070c90 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb733b9d640 1 -- 172.21.15.135:0/1553744236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c070c90 msgr2=0x7fb72c0d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb733b9d640 1 --2- 172.21.15.135:0/1553744236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c070c90 0x7fb72c0d0810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0097d0 tx=0x7fb71c02b930 comp rx=0 tx=0).stop 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb733b9d640 1 -- 172.21.15.135:0/1553744236 shutdown_connections 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb733b9d640 1 --2- 172.21.15.135:0/1553744236 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c070c90 0x7fb72c0d0810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb733b9d640 1 --2- 172.21.15.135:0/1553744236 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15f800 0x7fb72c070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb733b9d640 1 -- 172.21.15.135:0/1553744236 >> 172.21.15.135:0/1553744236 conn(0x7fb72c09a9f0 msgr2=0x7fb72c0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb733b9d640 1 -- 172.21.15.135:0/1553744236 shutdown_connections 2024-04-17T06:09:31.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.694+0000 7fb733b9d640 1 -- 172.21.15.135:0/1553744236 wait complete. 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.695+0000 7fb733b9d640 1 Processor -- start 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb733b9d640 1 -- start start 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb733b9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c070c90 0x7fb72c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb733b9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c15f800 0x7fb72c0b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb733b9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15fbe0 0x7fb72c0c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb733b9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c0cee10 con 0x7fb72c070c90 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb733b9d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c06c460 con 0x7fb72c15f800 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb733b9d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c0d0d50 con 0x7fb72c15fbe0 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb731939640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c070c90 0x7fb72c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb731939640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c070c90 0x7fb72c0b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39908/0 (socket says 172.21.15.135:39908) 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb731939640 1 -- 172.21.15.135:0/4184968134 learned_addr learned my addr 172.21.15.135:0/4184968134 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.696+0000 7fb731138640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c15f800 0x7fb72c0b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb73213a640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15fbe0 0x7fb72c0c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb731939640 1 -- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15fbe0 msgr2=0x7fb72c0c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb731939640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15fbe0 0x7fb72c0c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb731939640 1 -- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c15f800 msgr2=0x7fb72c0b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb731939640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c15f800 0x7fb72c0b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb731939640 1 -- 172.21.15.135:0/4184968134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c0b7180 con 0x7fb72c070c90 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb731138640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c15f800 0x7fb72c0b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb731939640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c070c90 0x7fb72c0b5010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb72800ebe0 tx=0x7fb72800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb7227fc640 1 -- 172.21.15.135:0/4184968134 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728017070 con 0x7fb72c070c90 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb7227fc640 1 -- 172.21.15.135:0/4184968134 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72800cc30 con 0x7fb72c070c90 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c0b5550 con 0x7fb72c070c90 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb7227fc640 1 -- 172.21.15.135:0/4184968134 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728013470 con 0x7fb72c070c90 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.697+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c0b4530 con 0x7fb72c070c90 2024-04-17T06:09:31.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.698+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c074940 con 0x7fb72c070c90 2024-04-17T06:09:31.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.698+0000 7fb7227fc640 1 -- 172.21.15.135:0/4184968134 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb72801a070 con 0x7fb72c070c90 2024-04-17T06:09:31.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.699+0000 7fb7227fc640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb71006c920 0x7fb71006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.699+0000 7fb7227fc640 1 -- 172.21.15.135:0/4184968134 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb72808bdc0 con 0x7fb72c070c90 2024-04-17T06:09:31.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.699+0000 7fb731138640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb71006c920 0x7fb71006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.699+0000 7fb731138640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb71006c920 0x7fb71006ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb71c036000 tx=0x7fb71c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.703+0000 7fb7227fc640 1 -- 172.21.15.135:0/4184968134 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb728059f40 con 0x7fb72c070c90 2024-04-17T06:09:31.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.876+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb72c073be0 con 0x7fb72c070c90 2024-04-17T06:09:31.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.877+0000 7fb7227fc640 1 -- 172.21.15.135:0/4184968134 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fb728059840 con 0x7fb72c070c90 2024-04-17T06:09:31.875 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:09:31.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb71006c920 msgr2=0x7fb71006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb71006c920 0x7fb71006ede0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb71c036000 tx=0x7fb71c008030 comp rx=0 tx=0).stop 2024-04-17T06:09:31.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c070c90 msgr2=0x7fb72c0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c070c90 0x7fb72c0b5010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb72800ebe0 tx=0x7fb72800c520 comp rx=0 tx=0).stop 2024-04-17T06:09:31.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 shutdown_connections 2024-04-17T06:09:31.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb72c15fbe0 0x7fb72c0c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb72c15f800 0x7fb72c0b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb71006c920 0x7fb71006ede0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 --2- 172.21.15.135:0/4184968134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb72c070c90 0x7fb72c0b5010 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 >> 172.21.15.135:0/4184968134 conn(0x7fb72c09a9f0 msgr2=0x7fb72c0b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:31.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 shutdown_connections 2024-04-17T06:09:31.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.879+0000 7fb733b9d640 1 -- 172.21.15.135:0/4184968134 wait complete. 2024-04-17T06:09:31.888 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:09:31.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.996+0000 7f24ccb8f640 1 Processor -- start 2024-04-17T06:09:31.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.996+0000 7f24ccb8f640 1 -- start start 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.997+0000 7f24ccb8f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c80d19c0 0x7f24c80cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.997+0000 7f24ccb8f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f24c80d2b00 0x7f24c81686f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.997+0000 7f24ccb8f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c80b7b00 con 0x7f24c815eae0 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.997+0000 7f24ccb8f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f24c80d2fb0 con 0x7f24c80d2b00 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.997+0000 7f24ccb8f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80d3200 con 0x7f24c80d19c0 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c6d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f24c80d2b00 0x7f24c81686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c659c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f24c815eae0 0x7f24c80d12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35840/0 (socket says 172.21.15.135:35840) 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c659c640 1 -- 172.21.15.135:0/1864315418 learned_addr learned my addr 172.21.15.135:0/1864315418 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:31.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c5d9b640 1 --2- 172.21.15.135:0/1864315418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c80d19c0 0x7f24c80cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:31.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c6d9d640 1 -- 172.21.15.135:0/1864315418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c80d19c0 msgr2=0x7f24c80cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:31.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c6d9d640 1 --2- 172.21.15.135:0/1864315418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c80d19c0 0x7f24c80cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c6d9d640 1 -- 172.21.15.135:0/1864315418 >> v1:172.21.15.135:6789/0 conn(0x7f24c815eae0 legacy=0x7f24c80d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c6d9d640 1 -- 172.21.15.135:0/1864315418 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80d34e0 con 0x7f24c80d2b00 2024-04-17T06:09:31.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.998+0000 7f24c559a640 1 -- 172.21.15.135:0/1864315418 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880017519 0 0) 0x7f24c80b7b00 con 0x7f24c815eae0 2024-04-17T06:09:31.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24c6d9d640 1 --2- 172.21.15.135:0/1864315418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c81686f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f24c00097a0 tx=0x7f24c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985bb08375d6b856 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:31.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24c559a640 1 -- 172.21.15.135:0/1864315418 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0034070 con 0x7f24c80d2b00 2024-04-17T06:09:31.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24c559a640 1 -- 172.21.15.135:0/1864315418 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c0030610 con 0x7f24c80d2b00 2024-04-17T06:09:31.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24c559a640 1 -- 172.21.15.135:0/1864315418 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00309e0 con 0x7f24c80d2b00 2024-04-17T06:09:31.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1864315418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 msgr2=0x7f24c81686f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:31.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1864315418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c81686f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f24c00097a0 tx=0x7f24c002b960 comp rx=0 tx=0).stop 2024-04-17T06:09:31.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1864315418 shutdown_connections 2024-04-17T06:09:31.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1864315418 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c80d19c0 0x7f24c80cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1864315418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c81686f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:31.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1864315418 >> 172.21.15.135:0/1864315418 conn(0x7f24c806ce10 msgr2=0x7f24c806d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:31.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1864315418 shutdown_connections 2024-04-17T06:09:31.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:31.999+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1864315418 wait complete. 2024-04-17T06:09:32.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.001+0000 7f24ccb8f640 1 Processor -- start 2024-04-17T06:09:32.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.001+0000 7f24ccb8f640 1 -- start start 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.001+0000 7f24ccb8f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c80d19c0 0x7f24c8155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.001+0000 7f24ccb8f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c80cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.001+0000 7f24ccb8f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c815eae0 0x7f24c815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.001+0000 7f24ccb8f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24c815eec0 con 0x7f24c80d2b00 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.001+0000 7f24ccb8f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80d3aa0 con 0x7f24c80d19c0 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.001+0000 7f24ccb8f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80b9ff0 con 0x7f24c815eae0 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c80cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c80cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39930/0 (socket says 172.21.15.135:39930) 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 -- 172.21.15.135:0/1721897052 learned_addr learned my addr 172.21.15.135:0/1721897052 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c659c640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c80d19c0 0x7f24c8155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c6d9d640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c815eae0 0x7f24c815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 -- 172.21.15.135:0/1721897052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c815eae0 msgr2=0x7f24c815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c815eae0 0x7f24c815ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 -- 172.21.15.135:0/1721897052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c80d19c0 msgr2=0x7f24c8155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c80d19c0 0x7f24c8155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 -- 172.21.15.135:0/1721897052 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80c9b70 con 0x7f24c80d2b00 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24c5d9b640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c80cac10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f24b000d830 tx=0x7f24b000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24beffd640 1 -- 172.21.15.135:0/1721897052 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b000b6c0 con 0x7f24c80d2b00 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24beffd640 1 -- 172.21.15.135:0/1721897052 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b0019070 con 0x7f24c80d2b00 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24beffd640 1 -- 172.21.15.135:0/1721897052 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b000ba80 con 0x7f24c80d2b00 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c80c9e50 con 0x7f24c80d2b00 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.002+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c80ca410 con 0x7f24c80d2b00 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.003+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c8068a20 con 0x7f24c80d2b00 2024-04-17T06:09:32.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.004+0000 7f24beffd640 1 -- 172.21.15.135:0/1721897052 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24b001e460 con 0x7f24c80d2b00 2024-04-17T06:09:32.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.004+0000 7f24beffd640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.004+0000 7f24beffd640 1 -- 172.21.15.135:0/1721897052 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f24b008c420 con 0x7f24c80d2b00 2024-04-17T06:09:32.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.004+0000 7f24c659c640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.005+0000 7f24c659c640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24a806c8f0 0x7f24a806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f24b80012e0 tx=0x7f24b8001630 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.008+0000 7f24beffd640 1 -- 172.21.15.135:0/1721897052 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24b0059ec0 con 0x7f24c80d2b00 2024-04-17T06:09:32.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.181+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f24c80d40c0 con 0x7f24c80d2b00 2024-04-17T06:09:32.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.181+0000 7f24beffd640 1 -- 172.21.15.135:0/1721897052 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f24b005e1c0 con 0x7f24c80d2b00 2024-04-17T06:09:32.179 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:32.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24a806c8f0 msgr2=0x7f24a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24a806c8f0 0x7f24a806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f24b80012e0 tx=0x7f24b8001630 comp rx=0 tx=0).stop 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 msgr2=0x7f24c80cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c80cac10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f24b000d830 tx=0x7f24b000dcf0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 shutdown_connections 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c815eae0 0x7f24c815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24a806c8f0 0x7f24a806edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24c80d2b00 0x7f24c80cac10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 --2- 172.21.15.135:0/1721897052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c80d19c0 0x7f24c8155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 >> 172.21.15.135:0/1721897052 conn(0x7f24c806ce10 msgr2=0x7f24c80cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 shutdown_connections 2024-04-17T06:09:32.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.183+0000 7f24ccb8f640 1 -- 172.21.15.135:0/1721897052 wait complete. 2024-04-17T06:09:32.192 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T06:09:32.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.299+0000 7fed1959d640 1 Processor -- start 2024-04-17T06:09:32.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.300+0000 7fed1959d640 1 -- start start 2024-04-17T06:09:32.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.300+0000 7fed1959d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fed140b0180 0x7fed14162400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.300+0000 7fed1959d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed140b0560 0x7fed140b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.300+0000 7fed1959d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed140d1e30 con 0x7fed140b63a0 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.300+0000 7fed1959d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fed140d1be0 con 0x7fed140b0180 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.300+0000 7fed1959d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed140b4270 con 0x7fed140b0560 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed18d9c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fed140b63a0 0x7fed1415f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35846/0 (socket says 172.21.15.135:35846) 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed18d9c640 1 -- 172.21.15.135:0/2691479365 learned_addr learned my addr 172.21.15.135:0/2691479365 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed13fff640 1 --2- 172.21.15.135:0/2691479365 >> v2:172.21.15.190:3300/0 conn(0x7fed140b0180 0x7fed14162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed0bfff640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed140b0560 0x7fed140b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 290501494 0 0) 0x7fed140d1e30 con 0x7fed140b63a0 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed00003520 con 0x7fed140b63a0 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2413459131 0 0) 0x7fed00003520 con 0x7fed140b63a0 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed140b0560 msgr2=0x7fed140b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed140b0560 0x7fed140b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 >> v2:172.21.15.190:3300/0 conn(0x7fed140b0180 msgr2=0x7fed14162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:32.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 --2- 172.21.15.135:0/2691479365 >> v2:172.21.15.190:3300/0 conn(0x7fed140b0180 0x7fed14162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed140d2110 con 0x7fed140b63a0 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fed0c0019f0 con 0x7fed140b63a0 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 >> v1:172.21.15.135:6789/0 conn(0x7fed140b63a0 legacy=0x7fed1415f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed000039b0 0x7fed00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed00004800 0x7fed00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed140b0560 0x7fed000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed140d1be0 con 0x7fed000039b0 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed140b4270 con 0x7fed00004800 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed0000ba40 con 0x7fed140b0560 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed13fff640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed000039b0 0x7fed00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed0bfff640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed00004800 0x7fed00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed18d9c640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed140b0560 0x7fed000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed13fff640 1 -- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed140b0560 msgr2=0x7fed000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:32.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed13fff640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed140b0560 0x7fed000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed13fff640 1 -- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed00004800 msgr2=0x7fed00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed13fff640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed00004800 0x7fed00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.301+0000 7fed13fff640 1 -- 172.21.15.135:0/2691479365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed00003520 con 0x7fed000039b0 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.302+0000 7fed13fff640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed000039b0 0x7fed00003dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed04009ac0 tx=0x7fed00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fcd46f3bccdc20 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.302+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04039070 con 0x7fed000039b0 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.302+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0402c710 con 0x7fed000039b0 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.302+0000 7fed137fe640 1 -- 172.21.15.135:0/2691479365 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04009690 con 0x7fed000039b0 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.303+0000 7fed1959d640 1 -- 172.21.15.135:0/2691479365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed000039b0 msgr2=0x7fed00003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.303+0000 7fed1959d640 1 --2- 172.21.15.135:0/2691479365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed000039b0 0x7fed00003dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed04009ac0 tx=0x7fed00005830 comp rx=0 tx=0).stop 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.303+0000 7fed18d9c640 1 -- 172.21.15.135:0/2691479365 reap_dead start 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.303+0000 7fed1959d640 1 -- 172.21.15.135:0/2691479365 shutdown_connections 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.303+0000 7fed1959d640 1 -- 172.21.15.135:0/2691479365 >> 172.21.15.135:0/2691479365 conn(0x7fed14073360 msgr2=0x7fed1409a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:32.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.303+0000 7fed1959d640 1 -- 172.21.15.135:0/2691479365 shutdown_connections 2024-04-17T06:09:32.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.303+0000 7fed1959d640 1 -- 172.21.15.135:0/2691479365 wait complete. 2024-04-17T06:09:32.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.304+0000 7fed1959d640 1 Processor -- start 2024-04-17T06:09:32.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.304+0000 7fed1959d640 1 -- start start 2024-04-17T06:09:32.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed1959d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000039b0 0x7fed1414fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed1959d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed141e80a0 0x7fed141e8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed1959d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed141f7b30 0x7fed141f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed1959d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed1415c870 con 0x7fed141f7b30 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed1959d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed140b4830 con 0x7fed000039b0 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed1959d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed14070c90 con 0x7fed141e80a0 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed18d9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed141f7b30 0x7fed141f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed18d9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed141f7b30 0x7fed141f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39952/0 (socket says 172.21.15.135:39952) 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed18d9c640 1 -- 172.21.15.135:0/1036957975 learned_addr learned my addr 172.21.15.135:0/1036957975 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed0bfff640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed141e80a0 0x7fed141e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed13fff640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000039b0 0x7fed1414fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed18d9c640 1 -- 172.21.15.135:0/1036957975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed141e80a0 msgr2=0x7fed141e8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed18d9c640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed141e80a0 0x7fed141e8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed18d9c640 1 -- 172.21.15.135:0/1036957975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000039b0 msgr2=0x7fed1414fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed18d9c640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000039b0 0x7fed1414fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.305+0000 7fed18d9c640 1 -- 172.21.15.135:0/1036957975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed141e8b60 con 0x7fed141f7b30 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.306+0000 7fed18d9c640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed141f7b30 0x7fed141f7fb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed0c002230 tx=0x7fed0c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.306+0000 7fed117fa640 1 -- 172.21.15.135:0/1036957975 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00ebb0 con 0x7fed141f7b30 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.306+0000 7fed117fa640 1 -- 172.21.15.135:0/1036957975 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c018070 con 0x7fed141f7b30 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.306+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed141e8e40 con 0x7fed141f7b30 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.306+0000 7fed117fa640 1 -- 172.21.15.135:0/1036957975 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c013420 con 0x7fed141f7b30 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.306+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed14157f00 con 0x7fed141f7b30 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.307+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed14157a10 con 0x7fed141f7b30 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.307+0000 7fed117fa640 1 -- 172.21.15.135:0/1036957975 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed0c0135c0 con 0x7fed141f7b30 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.308+0000 7fed117fa640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fecf406c8f0 0x7fecf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.308+0000 7fed13fff640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fecf406c8f0 0x7fecf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.308+0000 7fed117fa640 1 -- 172.21.15.135:0/1036957975 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fed0c08c020 con 0x7fed141f7b30 2024-04-17T06:09:32.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.308+0000 7fed13fff640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fecf406c8f0 0x7fecf406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed04036000 tx=0x7fed04030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.312+0000 7fed117fa640 1 -- 172.21.15.135:0/1036957975 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed0c059ac0 con 0x7fed141f7b30 2024-04-17T06:09:32.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.501+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fed14068a20 con 0x7fed141f7b30 2024-04-17T06:09:32.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.501+0000 7fed117fa640 1 -- 172.21.15.135:0/1036957975 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fed0c05ddc0 con 0x7fed141f7b30 2024-04-17T06:09:32.499 INFO:teuthology.orchestra.run.smithi135.stderr:listed 0 entries 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fecf406c8f0 msgr2=0x7fecf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fecf406c8f0 0x7fecf406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fed04036000 tx=0x7fed04030000 comp rx=0 tx=0).stop 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed141f7b30 msgr2=0x7fed141f7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed141f7b30 0x7fed141f7fb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed0c002230 tx=0x7fed0c00e530 comp rx=0 tx=0).stop 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 shutdown_connections 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fecf406c8f0 0x7fecf406edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed141f7b30 0x7fed141f7fb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed141e80a0 0x7fed141e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 --2- 172.21.15.135:0/1036957975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed000039b0 0x7fed1414fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 >> 172.21.15.135:0/1036957975 conn(0x7fed14073360 msgr2=0x7fed1415ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:32.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 shutdown_connections 2024-04-17T06:09:32.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.503+0000 7fed1959d640 1 -- 172.21.15.135:0/1036957975 wait complete. 2024-04-17T06:09:32.512 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:09:32.512 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:09:32.528 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:32.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.673+0000 7fdd183e2640 1 Processor -- start 2024-04-17T06:09:32.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.673+0000 7fdd183e2640 1 -- start start 2024-04-17T06:09:32.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd183e2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdd1009bd50 0x7fdd1009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd183e2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd1009af30 0x7fdd1006d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd183e2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd100d13f0 con 0x7fdd1015ed00 2024-04-17T06:09:32.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd183e2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdd100d19b0 con 0x7fdd1009bd50 2024-04-17T06:09:32.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd183e2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdd100d16d0 con 0x7fdd1009af30 2024-04-17T06:09:32.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd1697f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd1009af30 0x7fdd1006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd1597d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdd1009bd50 0x7fdd1009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd1697f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd1009af30 0x7fdd1006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:49042/0 (socket says 172.21.15.135:49042) 2024-04-17T06:09:32.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd1697f640 1 -- 172.21.15.135:0/2336947766 learned_addr learned my addr 172.21.15.135:0/2336947766 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:32.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd1597d640 1 -- 172.21.15.135:0/2336947766 >> v2:172.21.15.190:3300/0 conn(0x7fdd1009af30 msgr2=0x7fdd1006d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:32.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd1597d640 1 --2- 172.21.15.135:0/2336947766 >> v2:172.21.15.190:3300/0 conn(0x7fdd1009af30 0x7fdd1006d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd1597d640 1 -- 172.21.15.135:0/2336947766 >> v1:172.21.15.190:6789/0 conn(0x7fdd1015ed00 legacy=0x7fdd100d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.675+0000 7fdd1597d640 1 -- 172.21.15.135:0/2336947766 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1006cc20 con 0x7fdd1009bd50 2024-04-17T06:09:32.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.676+0000 7fdd1597d640 1 --2- 172.21.15.135:0/2336947766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1009bd50 0x7fdd1009a9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd1006ee10 tx=0x7fdd0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34457deaabbfd12 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.678+0000 7fdd1517c640 1 -- 172.21.15.135:0/2336947766 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00035020 con 0x7fdd1009bd50 2024-04-17T06:09:32.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.678+0000 7fdd1517c640 1 -- 172.21.15.135:0/2336947766 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd00031830 con 0x7fdd1009bd50 2024-04-17T06:09:32.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.678+0000 7fdd1517c640 1 -- 172.21.15.135:0/2336947766 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd00031c00 con 0x7fdd1009bd50 2024-04-17T06:09:32.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.678+0000 7fdd183e2640 1 -- 172.21.15.135:0/2336947766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1009bd50 msgr2=0x7fdd1009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.678+0000 7fdd183e2640 1 --2- 172.21.15.135:0/2336947766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1009bd50 0x7fdd1009a9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdd1006ee10 tx=0x7fdd0002b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.679+0000 7fdd183e2640 1 -- 172.21.15.135:0/2336947766 shutdown_connections 2024-04-17T06:09:32.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.679+0000 7fdd183e2640 1 --2- 172.21.15.135:0/2336947766 >> v2:172.21.15.190:3300/0 conn(0x7fdd1009af30 0x7fdd1006d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.679+0000 7fdd183e2640 1 --2- 172.21.15.135:0/2336947766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1009bd50 0x7fdd1009a9f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.679+0000 7fdd183e2640 1 -- 172.21.15.135:0/2336947766 >> 172.21.15.135:0/2336947766 conn(0x7fdd100b4190 msgr2=0x7fdd100b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:32.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.679+0000 7fdd183e2640 1 -- 172.21.15.135:0/2336947766 shutdown_connections 2024-04-17T06:09:32.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.679+0000 7fdd183e2640 1 -- 172.21.15.135:0/2336947766 wait complete. 2024-04-17T06:09:32.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.680+0000 7fdd183e2640 1 Processor -- start 2024-04-17T06:09:32.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.680+0000 7fdd183e2640 1 -- start start 2024-04-17T06:09:32.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd183e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1009af30 0x7fdd100cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd183e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1009bd50 0x7fdd101549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd183e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1015ed00 0x7fdd100cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd183e2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1006dae0 con 0x7fdd1015ed00 2024-04-17T06:09:32.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd183e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1006d960 con 0x7fdd1009af30 2024-04-17T06:09:32.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd183e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1006dc60 con 0x7fdd1009bd50 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd1697f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1015ed00 0x7fdd100cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd1697f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1015ed00 0x7fdd100cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39964/0 (socket says 172.21.15.135:39964) 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd1697f640 1 -- 172.21.15.135:0/4167595982 learned_addr learned my addr 172.21.15.135:0/4167595982 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd1597d640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1009bd50 0x7fdd101549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd1617e640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1009af30 0x7fdd100cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd1697f640 1 -- 172.21.15.135:0/4167595982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1009bd50 msgr2=0x7fdd101549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.681+0000 7fdd1697f640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1009bd50 0x7fdd101549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd1697f640 1 -- 172.21.15.135:0/4167595982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1009af30 msgr2=0x7fdd100cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd1697f640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1009af30 0x7fdd100cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd1697f640 1 -- 172.21.15.135:0/4167595982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd100c9c70 con 0x7fdd1015ed00 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd1697f640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1015ed00 0x7fdd100cc200 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009d90 tx=0x7fdd0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd06ffd640 1 -- 172.21.15.135:0/4167595982 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0800cd20 con 0x7fdd1015ed00 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd06ffd640 1 -- 172.21.15.135:0/4167595982 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08003990 con 0x7fdd1015ed00 2024-04-17T06:09:32.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd06ffd640 1 -- 172.21.15.135:0/4167595982 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08003d60 con 0x7fdd1015ed00 2024-04-17T06:09:32.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd100c9f50 con 0x7fdd1015ed00 2024-04-17T06:09:32.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.682+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd100ca510 con 0x7fdd1015ed00 2024-04-17T06:09:32.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.683+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd10069510 con 0x7fdd1015ed00 2024-04-17T06:09:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.683+0000 7fdd06ffd640 1 -- 172.21.15.135:0/4167595982 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd08012050 con 0x7fdd1015ed00 2024-04-17T06:09:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.684+0000 7fdd06ffd640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.684+0000 7fdd06ffd640 1 -- 172.21.15.135:0/4167595982 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdd0808c320 con 0x7fdd1015ed00 2024-04-17T06:09:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.684+0000 7fdd1617e640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.684+0000 7fdd1617e640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdcf406c8f0 0x7fdcf406edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007fd0 tx=0x7fdd0c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.688+0000 7fdd06ffd640 1 -- 172.21.15.135:0/4167595982 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd10069510 con 0x7fdd1015ed00 2024-04-17T06:09:32.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.864+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd10068a20 con 0x7fdd1015ed00 2024-04-17T06:09:32.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.865+0000 7fdd06ffd640 1 -- 172.21.15.135:0/4167595982 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7fdd08010040 con 0x7fdd1015ed00 2024-04-17T06:09:32.863 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:32.863 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:32.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdcf406c8f0 msgr2=0x7fdcf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdcf406c8f0 0x7fdcf406edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007fd0 tx=0x7fdd0c009040 comp rx=0 tx=0).stop 2024-04-17T06:09:32.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1015ed00 msgr2=0x7fdd100cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1015ed00 0x7fdd100cc200 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd08009d90 tx=0x7fdd0800c490 comp rx=0 tx=0).stop 2024-04-17T06:09:32.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 shutdown_connections 2024-04-17T06:09:32.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd1015ed00 0x7fdd100cc200 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1009bd50 0x7fdd101549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 --2- 172.21.15.135:0/4167595982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd1009af30 0x7fdd100cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 >> 172.21.15.135:0/4167595982 conn(0x7fdd100b4190 msgr2=0x7fdd100719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:32.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 shutdown_connections 2024-04-17T06:09:32.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.867+0000 7fdd183e2640 1 -- 172.21.15.135:0/4167595982 wait complete. 2024-04-17T06:09:32.876 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:09:32.877 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:32.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.987+0000 7fc75e104640 1 Processor -- start 2024-04-17T06:09:32.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.987+0000 7fc75e104640 1 -- start start 2024-04-17T06:09:32.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.987+0000 7fc75e104640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc75815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.987+0000 7fc75e104640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc75806f610 0x7fc75815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.987+0000 7fc75e104640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc75815fb80 con 0x7fc758158a50 2024-04-17T06:09:32.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.987+0000 7fc75e104640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc75815b840 con 0x7fc75806f610 2024-04-17T06:09:32.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.987+0000 7fc75e104640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc75806fac0 con 0x7fc758158e30 2024-04-17T06:09:32.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc7577fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc758158a50 0x7fc7580b3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35856/0 (socket says 172.21.15.135:35856) 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc7577fe640 1 -- 172.21.15.135:0/3947778606 learned_addr learned my addr 172.21.15.135:0/3947778606 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc756ffd640 1 --2- 172.21.15.135:0/3947778606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc75815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc757fff640 1 --2- 172.21.15.135:0/3947778606 >> v2:172.21.15.190:3300/0 conn(0x7fc75806f610 0x7fc75815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc7567fc640 1 -- 172.21.15.135:0/3947778606 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4110537021 0 0) 0x7fc75815fb80 con 0x7fc758158a50 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc7567fc640 1 -- 172.21.15.135:0/3947778606 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc740003520 con 0x7fc758158a50 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc756ffd640 1 -- 172.21.15.135:0/3947778606 >> v2:172.21.15.190:3300/0 conn(0x7fc75806f610 msgr2=0x7fc75815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc756ffd640 1 --2- 172.21.15.135:0/3947778606 >> v2:172.21.15.190:3300/0 conn(0x7fc75806f610 0x7fc75815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc756ffd640 1 -- 172.21.15.135:0/3947778606 >> v1:172.21.15.135:6789/0 conn(0x7fc758158a50 legacy=0x7fc7580b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.988+0000 7fc756ffd640 1 -- 172.21.15.135:0/3947778606 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75815f2e0 con 0x7fc758158e30 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.989+0000 7fc756ffd640 1 --2- 172.21.15.135:0/3947778606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc75815ec20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0097a0 tx=0x7fc74c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470f9f8689273fde server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.990+0000 7fc7567fc640 1 -- 172.21.15.135:0/3947778606 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c035020 con 0x7fc758158e30 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.990+0000 7fc7567fc640 1 -- 172.21.15.135:0/3947778606 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74c031810 con 0x7fc758158e30 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.990+0000 7fc7567fc640 1 -- 172.21.15.135:0/3947778606 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c031be0 con 0x7fc758158e30 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.990+0000 7fc75e104640 1 -- 172.21.15.135:0/3947778606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 msgr2=0x7fc75815ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.990+0000 7fc75e104640 1 --2- 172.21.15.135:0/3947778606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc75815ec20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0097a0 tx=0x7fc74c02b8b0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.991+0000 7fc75e104640 1 -- 172.21.15.135:0/3947778606 shutdown_connections 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.991+0000 7fc75e104640 1 --2- 172.21.15.135:0/3947778606 >> v2:172.21.15.190:3300/0 conn(0x7fc75806f610 0x7fc75815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.991+0000 7fc75e104640 1 --2- 172.21.15.135:0/3947778606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc75815ec20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.991+0000 7fc75e104640 1 -- 172.21.15.135:0/3947778606 >> 172.21.15.135:0/3947778606 conn(0x7fc7580d0520 msgr2=0x7fc7580d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:32.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.991+0000 7fc75e104640 1 -- 172.21.15.135:0/3947778606 shutdown_connections 2024-04-17T06:09:32.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.991+0000 7fc75e104640 1 -- 172.21.15.135:0/3947778606 wait complete. 2024-04-17T06:09:32.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.992+0000 7fc75e104640 1 Processor -- start 2024-04-17T06:09:32.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.992+0000 7fc75e104640 1 -- start start 2024-04-17T06:09:32.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc75e104640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc75806f610 0x7fc75815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc75e104640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758158a50 0x7fc75814af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc75e104640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc7581e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc75e104640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc7580b3ca0 con 0x7fc75806f610 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc75e104640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc75815f8a0 con 0x7fc758158a50 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc75e104640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc7581f4720 con 0x7fc758158e30 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc75806f610 0x7fc75815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc75806f610 0x7fc75815c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39970/0 (socket says 172.21.15.135:39970) 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc7577fe640 1 -- 172.21.15.135:0/2688760949 learned_addr learned my addr 172.21.15.135:0/2688760949 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc757fff640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc7581e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc756ffd640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758158a50 0x7fc75814af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc7577fe640 1 -- 172.21.15.135:0/2688760949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 msgr2=0x7fc7581e58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc7577fe640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc7581e58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc7577fe640 1 -- 172.21.15.135:0/2688760949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758158a50 msgr2=0x7fc75814af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc7577fe640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758158a50 0x7fc75814af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:32.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.993+0000 7fc7577fe640 1 -- 172.21.15.135:0/2688760949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75815bd70 con 0x7fc75806f610 2024-04-17T06:09:32.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.994+0000 7fc7577fe640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc75806f610 0x7fc75815c850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc74800cb40 tx=0x7fc74800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.994+0000 7fc73ffff640 1 -- 172.21.15.135:0/2688760949 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748017070 con 0x7fc75806f610 2024-04-17T06:09:32.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.994+0000 7fc73ffff640 1 -- 172.21.15.135:0/2688760949 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74800aca0 con 0x7fc75806f610 2024-04-17T06:09:32.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.994+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75815c050 con 0x7fc75806f610 2024-04-17T06:09:32.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.994+0000 7fc73ffff640 1 -- 172.21.15.135:0/2688760949 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748012620 con 0x7fc75806f610 2024-04-17T06:09:32.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.994+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75814b4a0 con 0x7fc75806f610 2024-04-17T06:09:32.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.995+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7580cd510 con 0x7fc75806f610 2024-04-17T06:09:32.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.995+0000 7fc73ffff640 1 -- 172.21.15.135:0/2688760949 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc74801a070 con 0x7fc75806f610 2024-04-17T06:09:32.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.996+0000 7fc73ffff640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc74406c8f0 0x7fc74406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:32.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.996+0000 7fc73ffff640 1 -- 172.21.15.135:0/2688760949 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc74808bc00 con 0x7fc75806f610 2024-04-17T06:09:32.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.996+0000 7fc756ffd640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc74406c8f0 0x7fc74406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:32.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:32.996+0000 7fc756ffd640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc74406c8f0 0x7fc74406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc74c036000 tx=0x7fc74c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:32.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.000+0000 7fc73ffff640 1 -- 172.21.15.135:0/2688760949 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7580cd510 con 0x7fc75806f610 2024-04-17T06:09:33.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.176+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc758074940 con 0x7fc75806f610 2024-04-17T06:09:33.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.177+0000 7fc73ffff640 1 -- 172.21.15.135:0/2688760949 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fc748092080 con 0x7fc75806f610 2024-04-17T06:09:33.175 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.178+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc74406c8f0 msgr2=0x7fc74406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.178+0000 7fc75e104640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc74406c8f0 0x7fc74406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc74c036000 tx=0x7fc74c03c040 comp rx=0 tx=0).stop 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.178+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc75806f610 msgr2=0x7fc75815c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.178+0000 7fc75e104640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc75806f610 0x7fc75815c850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc74800cb40 tx=0x7fc74800a060 comp rx=0 tx=0).stop 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.178+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 shutdown_connections 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.179+0000 7fc75e104640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc758158e30 0x7fc7581e58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.179+0000 7fc75e104640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc758158a50 0x7fc75814af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.179+0000 7fc75e104640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc74406c8f0 0x7fc74406edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.179+0000 7fc75e104640 1 --2- 172.21.15.135:0/2688760949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc75806f610 0x7fc75815c850 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.179+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 >> 172.21.15.135:0/2688760949 conn(0x7fc7580d0520 msgr2=0x7fc7580b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:33.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.179+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 shutdown_connections 2024-04-17T06:09:33.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.179+0000 7fc75e104640 1 -- 172.21.15.135:0/2688760949 wait complete. 2024-04-17T06:09:33.188 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:09:33.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.296+0000 7fd8ce7e2640 1 Processor -- start 2024-04-17T06:09:33.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.296+0000 7fd8ce7e2640 1 -- start start 2024-04-17T06:09:33.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8ce7e2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd8c809bbe0 0x7fd8c809b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8ce7e2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c806db80 con 0x7fd8c815ee30 2024-04-17T06:09:33.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8ce7e2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c806de60 con 0x7fd8c80cd630 2024-04-17T06:09:33.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8ce7e2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c80cce30 con 0x7fd8c809bbe0 2024-04-17T06:09:33.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8bffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd8c809bbe0 0x7fd8c809b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8c7fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd8c815ee30 0x7fd8c809a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35864/0 (socket says 172.21.15.135:35864) 2024-04-17T06:09:33.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8bffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd8c809bbe0 0x7fd8c809b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39978/0 (socket says 172.21.15.135:39978) 2024-04-17T06:09:33.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8c7fff640 1 -- 172.21.15.135:0/1253926212 learned_addr learned my addr 172.21.15.135:0/1253926212 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:33.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8bffff640 1 -- 172.21.15.135:0/1253926212 >> v1:172.21.15.190:6789/0 conn(0x7fd8c80cd630 legacy=0x7fd8c80cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:33.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8bffff640 1 -- 172.21.15.135:0/1253926212 >> v1:172.21.15.135:6789/0 conn(0x7fd8c815ee30 legacy=0x7fd8c809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.297+0000 7fd8bffff640 1 -- 172.21.15.135:0/1253926212 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c80cd110 con 0x7fd8c809bbe0 2024-04-17T06:09:33.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.298+0000 7fd8bffff640 1 --2- 172.21.15.135:0/1253926212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c809b100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00097a0 tx=0x7fd8b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbab15a44b8a3cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.298+0000 7fd8c77fe640 1 -- 172.21.15.135:0/1253926212 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0035020 con 0x7fd8c809bbe0 2024-04-17T06:09:33.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.298+0000 7fd8c77fe640 1 -- 172.21.15.135:0/1253926212 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b0031830 con 0x7fd8c809bbe0 2024-04-17T06:09:33.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.298+0000 7fd8c77fe640 1 -- 172.21.15.135:0/1253926212 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0031c00 con 0x7fd8c809bbe0 2024-04-17T06:09:33.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.299+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1253926212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 msgr2=0x7fd8c809b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.299+0000 7fd8ce7e2640 1 --2- 172.21.15.135:0/1253926212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c809b100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00097a0 tx=0x7fd8b002b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.299+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1253926212 shutdown_connections 2024-04-17T06:09:33.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.299+0000 7fd8ce7e2640 1 --2- 172.21.15.135:0/1253926212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c809b100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.299+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1253926212 >> 172.21.15.135:0/1253926212 conn(0x7fd8c80b41a0 msgr2=0x7fd8c80b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:33.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.299+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1253926212 shutdown_connections 2024-04-17T06:09:33.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.299+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1253926212 wait complete. 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.300+0000 7fd8ce7e2640 1 Processor -- start 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.300+0000 7fd8ce7e2640 1 -- start start 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ce7e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c80c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ce7e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8c80cd630 0x7fd8c8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ce7e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8c815ee30 0x7fd8c8156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ce7e2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c806d910 con 0x7fd8c809bbe0 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ce7e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c806d790 con 0x7fd8c80cd630 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ce7e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c809b640 con 0x7fd8c815ee30 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c80c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c80c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39988/0 (socket says 172.21.15.135:39988) 2024-04-17T06:09:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 -- 172.21.15.135:0/1198883560 learned_addr learned my addr 172.21.15.135:0/1198883560 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8bffff640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8c80cd630 0x7fd8c8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ccd7f640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8c815ee30 0x7fd8c8156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 -- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8c815ee30 msgr2=0x7fd8c8156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8c815ee30 0x7fd8c8156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 -- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8c80cd630 msgr2=0x7fd8c8155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8c80cd630 0x7fd8c8155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 -- 172.21.15.135:0/1198883560 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c80b7e60 con 0x7fd8c809bbe0 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ccd7f640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8c815ee30 0x7fd8c8156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c7fff640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c80c8c00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800c430 tx=0x7fd8b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c57fa640 1 -- 172.21.15.135:0/1198883560 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8017070 con 0x7fd8c809bbe0 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c57fa640 1 -- 172.21.15.135:0/1198883560 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b8005ce0 con 0x7fd8c809bbe0 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c80b8140 con 0x7fd8c809bbe0 2024-04-17T06:09:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8c57fa640 1 -- 172.21.15.135:0/1198883560 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80072f0 con 0x7fd8c809bbe0 2024-04-17T06:09:33.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.301+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c80c9140 con 0x7fd8c809bbe0 2024-04-17T06:09:33.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.302+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c8074940 con 0x7fd8c809bbe0 2024-04-17T06:09:33.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.303+0000 7fd8c57fa640 1 -- 172.21.15.135:0/1198883560 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8b801a070 con 0x7fd8c809bbe0 2024-04-17T06:09:33.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.303+0000 7fd8c57fa640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd8a806c8f0 0x7fd8a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.303+0000 7fd8c57fa640 1 -- 172.21.15.135:0/1198883560 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd8b808b170 con 0x7fd8c809bbe0 2024-04-17T06:09:33.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.303+0000 7fd8bffff640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd8a806c8f0 0x7fd8a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.304+0000 7fd8bffff640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd8a806c8f0 0x7fd8a806edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0036000 tx=0x7fd8b003c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.307+0000 7fd8c57fa640 1 -- 172.21.15.135:0/1198883560 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8b8058ca0 con 0x7fd8c809bbe0 2024-04-17T06:09:33.482 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.484+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd8c80cf440 con 0x7fd8c809bbe0 2024-04-17T06:09:33.482 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.484+0000 7fd8c57fa640 1 -- 172.21.15.135:0/1198883560 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fd8b805cfa0 con 0x7fd8c809bbe0 2024-04-17T06:09:33.482 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd8a806c8f0 msgr2=0x7fd8a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd8a806c8f0 0x7fd8a806edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0036000 tx=0x7fd8b003c040 comp rx=0 tx=0).stop 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 msgr2=0x7fd8c80c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c80c8c00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800c430 tx=0x7fd8b800c8f0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 shutdown_connections 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8c815ee30 0x7fd8c8156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8c80cd630 0x7fd8c8155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd8a806c8f0 0x7fd8a806edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 --2- 172.21.15.135:0/1198883560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8c809bbe0 0x7fd8c80c8c00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 >> 172.21.15.135:0/1198883560 conn(0x7fd8c80b41a0 msgr2=0x7fd8c806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 shutdown_connections 2024-04-17T06:09:33.485 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.486+0000 7fd8ce7e2640 1 -- 172.21.15.135:0/1198883560 wait complete. 2024-04-17T06:09:33.495 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:09:33.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.604+0000 7fe950050640 1 Processor -- start 2024-04-17T06:09:33.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.604+0000 7fe950050640 1 -- start start 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe950050640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe94815ee30 0x7fe94809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe950050640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe9480cd630 0x7fe9480cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe950050640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe94806db80 con 0x7fe94809bbe0 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe950050640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe94806de60 con 0x7fe9480cd630 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe950050640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe9480cce30 con 0x7fe94815ee30 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94e5ed640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe9480cd630 0x7fe9480cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94e5ed640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe9480cd630 0x7fe9480cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:39996/0 (socket says 172.21.15.135:39996) 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94e5ed640 1 -- 172.21.15.135:0/1342538608 learned_addr learned my addr 172.21.15.135:0/1342538608 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94d5eb640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe94809bbe0 0x7fe94809af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35866/0 (socket says 172.21.15.135:35866) 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94ddec640 1 --2- 172.21.15.135:0/1342538608 >> v2:172.21.15.190:3300/0 conn(0x7fe94815ee30 0x7fe94809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94e5ed640 1 -- 172.21.15.135:0/1342538608 >> v2:172.21.15.190:3300/0 conn(0x7fe94815ee30 msgr2=0x7fe94809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94e5ed640 1 --2- 172.21.15.135:0/1342538608 >> v2:172.21.15.190:3300/0 conn(0x7fe94815ee30 0x7fe94809a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94e5ed640 1 -- 172.21.15.135:0/1342538608 >> v1:172.21.15.135:6789/0 conn(0x7fe94809bbe0 legacy=0x7fe94809af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94e5ed640 1 -- 172.21.15.135:0/1342538608 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9480cd110 con 0x7fe9480cd630 2024-04-17T06:09:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.605+0000 7fe94cdea640 1 -- 172.21.15.135:0/1342538608 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804226909 0 0) 0x7fe94806db80 con 0x7fe94809bbe0 2024-04-17T06:09:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.606+0000 7fe94e5ed640 1 --2- 172.21.15.135:0/1342538608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe9480cda60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe948069720 tx=0x7fe94002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd285f94d6ccc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.606+0000 7fe94cdea640 1 -- 172.21.15.135:0/1342538608 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94002cb20 con 0x7fe9480cd630 2024-04-17T06:09:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.606+0000 7fe94cdea640 1 -- 172.21.15.135:0/1342538608 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94002ccc0 con 0x7fe9480cd630 2024-04-17T06:09:33.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.606+0000 7fe94cdea640 1 -- 172.21.15.135:0/1342538608 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940034610 con 0x7fe9480cd630 2024-04-17T06:09:33.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.607+0000 7fe950050640 1 -- 172.21.15.135:0/1342538608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 msgr2=0x7fe9480cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.607+0000 7fe950050640 1 --2- 172.21.15.135:0/1342538608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe9480cda60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe948069720 tx=0x7fe94002bf80 comp rx=0 tx=0).stop 2024-04-17T06:09:33.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.607+0000 7fe950050640 1 -- 172.21.15.135:0/1342538608 shutdown_connections 2024-04-17T06:09:33.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.607+0000 7fe950050640 1 --2- 172.21.15.135:0/1342538608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe9480cda60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.607+0000 7fe950050640 1 --2- 172.21.15.135:0/1342538608 >> v2:172.21.15.190:3300/0 conn(0x7fe94815ee30 0x7fe94809a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.607+0000 7fe950050640 1 -- 172.21.15.135:0/1342538608 >> 172.21.15.135:0/1342538608 conn(0x7fe9480b41a0 msgr2=0x7fe9480b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.607+0000 7fe950050640 1 -- 172.21.15.135:0/1342538608 shutdown_connections 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.607+0000 7fe950050640 1 -- 172.21.15.135:0/1342538608 wait complete. 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.608+0000 7fe950050640 1 Processor -- start 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.608+0000 7fe950050640 1 -- start start 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe950050640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94809bbe0 0x7fe94815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe950050640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe948159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe950050640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94815ee30 0x7fe9480cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe950050640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe94806d910 con 0x7fe9480cd630 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe950050640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe94806d790 con 0x7fe94809bbe0 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe950050640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe94809b640 con 0x7fe94815ee30 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe948159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe948159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40006/0 (socket says 172.21.15.135:40006) 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 -- 172.21.15.135:0/312357660 learned_addr learned my addr 172.21.15.135:0/312357660 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94ddec640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94809bbe0 0x7fe94815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94e5ed640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94815ee30 0x7fe9480cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 -- 172.21.15.135:0/312357660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94815ee30 msgr2=0x7fe9480cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94815ee30 0x7fe9480cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 -- 172.21.15.135:0/312357660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94809bbe0 msgr2=0x7fe94815fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94809bbe0 0x7fe94815fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 -- 172.21.15.135:0/312357660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe948160550 con 0x7fe9480cd630 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe94d5eb640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe948159e50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe93800c4d0 tx=0x7fe93800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe93e7fc640 1 -- 172.21.15.135:0/312357660 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938018070 con 0x7fe9480cd630 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.609+0000 7fe93e7fc640 1 -- 172.21.15.135:0/312357660 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe938007500 con 0x7fe9480cd630 2024-04-17T06:09:33.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.610+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe948160830 con 0x7fe9480cd630 2024-04-17T06:09:33.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.610+0000 7fe93e7fc640 1 -- 172.21.15.135:0/312357660 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938004a20 con 0x7fe9480cd630 2024-04-17T06:09:33.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.610+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9480cb790 con 0x7fe9480cd630 2024-04-17T06:09:33.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.611+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe948074940 con 0x7fe9480cd630 2024-04-17T06:09:33.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.611+0000 7fe93e7fc640 1 -- 172.21.15.135:0/312357660 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe93801b070 con 0x7fe9480cd630 2024-04-17T06:09:33.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.612+0000 7fe93e7fc640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe92c06c8f0 0x7fe92c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.612+0000 7fe94ddec640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe92c06c8f0 0x7fe92c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.612+0000 7fe93e7fc640 1 -- 172.21.15.135:0/312357660 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe93808b0b0 con 0x7fe9480cd630 2024-04-17T06:09:33.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.612+0000 7fe94ddec640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe92c06c8f0 0x7fe92c06edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe9440079a0 tx=0x7fe944008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.616+0000 7fe93e7fc640 1 -- 172.21.15.135:0/312357660 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe938058be0 con 0x7fe9480cd630 2024-04-17T06:09:33.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.794+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe9480cf640 con 0x7fe9480cd630 2024-04-17T06:09:33.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.794+0000 7fe93e7fc640 1 -- 172.21.15.135:0/312357660 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fe93805cee0 con 0x7fe9480cd630 2024-04-17T06:09:33.793 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe92c06c8f0 msgr2=0x7fe92c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe92c06c8f0 0x7fe92c06edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe9440079a0 tx=0x7fe944008040 comp rx=0 tx=0).stop 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 msgr2=0x7fe948159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe948159e50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe93800c4d0 tx=0x7fe93800c990 comp rx=0 tx=0).stop 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 shutdown_connections 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe94815ee30 0x7fe9480cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe92c06c8f0 0x7fe92c06edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9480cd630 0x7fe948159e50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 --2- 172.21.15.135:0/312357660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe94809bbe0 0x7fe94815fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 >> 172.21.15.135:0/312357660 conn(0x7fe9480b41a0 msgr2=0x7fe94809a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 shutdown_connections 2024-04-17T06:09:33.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.796+0000 7fe950050640 1 -- 172.21.15.135:0/312357660 wait complete. 2024-04-17T06:09:33.805 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:09:33.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.912+0000 7f0b8822a640 1 Processor -- start 2024-04-17T06:09:33.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.913+0000 7f0b8822a640 1 -- start start 2024-04-17T06:09:33.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.913+0000 7f0b8822a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015f800 0x7f0b80070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.913+0000 7f0b8822a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0b80070c90 0x7f0b800d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.913+0000 7f0b8822a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0b8015fbe0 0x7f0b800d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b8822a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8006c8f0 con 0x7f0b80070c90 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b8822a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8006f070 con 0x7f0b8015fbe0 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b8822a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8006c460 con 0x7f0b8015f800 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b857c5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0b80070c90 0x7f0b800d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b857c5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0b80070c90 0x7f0b800d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40018/0 (socket says 172.21.15.135:40018) 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b857c5640 1 -- 172.21.15.135:0/2002093444 learned_addr learned my addr 172.21.15.135:0/2002093444 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b85fc6640 1 --2- 172.21.15.135:0/2002093444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015f800 0x7f0b80070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b867c7640 1 --2- 172.21.15.135:0/2002093444 >> v2:172.21.15.190:3300/0 conn(0x7f0b8015fbe0 0x7f0b800d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b857c5640 1 -- 172.21.15.135:0/2002093444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015f800 msgr2=0x7f0b80070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b857c5640 1 --2- 172.21.15.135:0/2002093444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015f800 0x7f0b80070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b857c5640 1 -- 172.21.15.135:0/2002093444 >> v2:172.21.15.190:3300/0 conn(0x7f0b8015fbe0 msgr2=0x7f0b800d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b857c5640 1 --2- 172.21.15.135:0/2002093444 >> v2:172.21.15.190:3300/0 conn(0x7f0b8015fbe0 0x7f0b800d0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.914+0000 7f0b857c5640 1 -- 172.21.15.135:0/2002093444 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8006bb30 con 0x7f0b80070c90 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b857c5640 1 --2- 172.21.15.135:0/2002093444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b80070c90 0x7f0b800d0810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b8009afa0 tx=0x7f0b7402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525d658999f61154 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b84fc4640 1 -- 172.21.15.135:0/2002093444 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7402cb00 con 0x7f0b80070c90 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b84fc4640 1 -- 172.21.15.135:0/2002093444 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7402cca0 con 0x7f0b80070c90 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b84fc4640 1 -- 172.21.15.135:0/2002093444 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74034610 con 0x7f0b80070c90 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b8822a640 1 -- 172.21.15.135:0/2002093444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b80070c90 msgr2=0x7f0b800d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2002093444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b80070c90 0x7f0b800d0810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b8009afa0 tx=0x7f0b7402bf60 comp rx=0 tx=0).stop 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b8822a640 1 -- 172.21.15.135:0/2002093444 shutdown_connections 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2002093444 >> v2:172.21.15.190:3300/0 conn(0x7f0b8015fbe0 0x7f0b800d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2002093444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b80070c90 0x7f0b800d0810 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2002093444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015f800 0x7f0b80070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.915+0000 7f0b8822a640 1 -- 172.21.15.135:0/2002093444 >> 172.21.15.135:0/2002093444 conn(0x7f0b8009a9f0 msgr2=0x7f0b800b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:33.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.916+0000 7f0b8822a640 1 -- 172.21.15.135:0/2002093444 shutdown_connections 2024-04-17T06:09:33.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.916+0000 7f0b8822a640 1 -- 172.21.15.135:0/2002093444 wait complete. 2024-04-17T06:09:33.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b8822a640 1 Processor -- start 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b8822a640 1 -- start start 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b8822a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80070c90 0x7f0b80154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b8822a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b8015f800 0x7f0b800c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b8822a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015fbe0 0x7f0b80160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b8822a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80162530 con 0x7f0b8015f800 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b8822a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8006c0f0 con 0x7f0b80070c90 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b8822a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0b801626b0 con 0x7f0b8015fbe0 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b857c5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b8015f800 0x7f0b800c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b857c5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b8015f800 0x7f0b800c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40034/0 (socket says 172.21.15.135:40034) 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.917+0000 7f0b857c5640 1 -- 172.21.15.135:0/2233155679 learned_addr learned my addr 172.21.15.135:0/2233155679 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b867c7640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015fbe0 0x7f0b80160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b85fc6640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80070c90 0x7f0b80154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b857c5640 1 -- 172.21.15.135:0/2233155679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015fbe0 msgr2=0x7f0b80160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b857c5640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015fbe0 0x7f0b80160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b857c5640 1 -- 172.21.15.135:0/2233155679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80070c90 msgr2=0x7f0b80154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b857c5640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80070c90 0x7f0b80154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b857c5640 1 -- 172.21.15.135:0/2233155679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8015d9a0 con 0x7f0b8015f800 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b857c5640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b8015f800 0x7f0b800c6480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b74036040 tx=0x7f0b74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b7e7fc640 1 -- 172.21.15.135:0/2233155679 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74043020 con 0x7f0b8015f800 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b7e7fc640 1 -- 172.21.15.135:0/2233155679 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7402ce50 con 0x7f0b8015f800 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8015dc80 con 0x7f0b8015f800 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b7e7fc640 1 -- 172.21.15.135:0/2233155679 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74009510 con 0x7f0b8015f800 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.918+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8015e240 con 0x7f0b8015f800 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.919+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b80074940 con 0x7f0b8015f800 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.920+0000 7f0b7e7fc640 1 -- 172.21.15.135:0/2233155679 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b74046020 con 0x7f0b8015f800 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.920+0000 7f0b7e7fc640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:33.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.920+0000 7f0b7e7fc640 1 -- 172.21.15.135:0/2233155679 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0b740ae280 con 0x7f0b8015f800 2024-04-17T06:09:33.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.920+0000 7f0b85fc6640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:33.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.920+0000 7f0b85fc6640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009ac0 tx=0x7f0b78009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:33.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:33.924+0000 7f0b7e7fc640 1 -- 172.21.15.135:0/2233155679 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b7404d030 con 0x7f0b8015f800 2024-04-17T06:09:34.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.099+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0b801620f0 con 0x7f0b8015f800 2024-04-17T06:09:34.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.099+0000 7f0b7e7fc640 1 -- 172.21.15.135:0/2233155679 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f0b74082070 con 0x7f0b8015f800 2024-04-17T06:09:34.097 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:34.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0b6c06c8f0 msgr2=0x7f0b6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009ac0 tx=0x7f0b78009190 comp rx=0 tx=0).stop 2024-04-17T06:09:34.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b8015f800 msgr2=0x7f0b800c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.099 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b8015f800 0x7f0b800c6480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b74036040 tx=0x7f0b74004a40 comp rx=0 tx=0).stop 2024-04-17T06:09:34.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 shutdown_connections 2024-04-17T06:09:34.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0b8015fbe0 0x7f0b80160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0b6c06c8f0 0x7f0b6c06edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b8015f800 0x7f0b800c6480 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 --2- 172.21.15.135:0/2233155679 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0b80070c90 0x7f0b80154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 >> 172.21.15.135:0/2233155679 conn(0x7f0b8009a9f0 msgr2=0x7f0b8006fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:34.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 shutdown_connections 2024-04-17T06:09:34.100 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.101+0000 7f0b8822a640 1 -- 172.21.15.135:0/2233155679 wait complete. 2024-04-17T06:09:34.111 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T06:09:34.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.217+0000 7f814e80c640 1 Processor -- start 2024-04-17T06:09:34.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.217+0000 7f814e80c640 1 -- start start 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.217+0000 7f814e80c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81480d19c0 0x7f81480cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.217+0000 7f814e80c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81480b7b00 con 0x7f814815eae0 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.217+0000 7f814e80c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81480d3200 con 0x7f81480d2b00 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.217+0000 7f814e80c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f81480d2fb0 con 0x7f81480d19c0 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f8147fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f814815eae0 0x7f81480d12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:35870/0 (socket says 172.21.15.135:35870) 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f8147fff640 1 -- 172.21.15.135:0/1605360176 learned_addr learned my addr 172.21.15.135:0/1605360176 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f81477fe640 1 --2- 172.21.15.135:0/1605360176 >> v2:172.21.15.190:3300/0 conn(0x7f81480d19c0 0x7f81480cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f8146ffd640 1 -- 172.21.15.135:0/1605360176 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1569776827 0 0) 0x7f81480b7b00 con 0x7f814815eae0 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f8146ffd640 1 -- 172.21.15.135:0/1605360176 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8130003520 con 0x7f814815eae0 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f8146ffd640 1 -- 172.21.15.135:0/1605360176 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357402300 0 0) 0x7f81480d3200 con 0x7f81480d2b00 2024-04-17T06:09:34.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f8146ffd640 1 -- 172.21.15.135:0/1605360176 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81480b7b00 con 0x7f81480d2b00 2024-04-17T06:09:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f81477fe640 1 -- 172.21.15.135:0/1605360176 >> v1:172.21.15.190:6789/0 conn(0x7f81480d2b00 legacy=0x7f81481686f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f8146ffd640 1 -- 172.21.15.135:0/1605360176 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3471973048 0 0) 0x7f8130003520 con 0x7f814815eae0 2024-04-17T06:09:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f81477fe640 1 -- 172.21.15.135:0/1605360176 >> v1:172.21.15.135:6789/0 conn(0x7f814815eae0 legacy=0x7f81480d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.218+0000 7f81477fe640 1 -- 172.21.15.135:0/1605360176 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81480d34e0 con 0x7f81480d19c0 2024-04-17T06:09:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.219+0000 7f81477fe640 1 --2- 172.21.15.135:0/1605360176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81480d19c0 0x7f81480cfb10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f813c0097a0 tx=0x7f813c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1a5bc51096b340 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.219+0000 7f8146ffd640 1 -- 172.21.15.135:0/1605360176 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138001670 con 0x7f81480d19c0 2024-04-17T06:09:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.219+0000 7f8146ffd640 1 -- 172.21.15.135:0/1605360176 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813c037070 con 0x7f81480d19c0 2024-04-17T06:09:34.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.219+0000 7f8146ffd640 1 -- 172.21.15.135:0/1605360176 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0317b0 con 0x7f81480d19c0 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.220+0000 7f814e80c640 1 -- 172.21.15.135:0/1605360176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81480d19c0 msgr2=0x7f81480cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.220+0000 7f814e80c640 1 --2- 172.21.15.135:0/1605360176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81480d19c0 0x7f81480cfb10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f813c0097a0 tx=0x7f813c02b8b0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.220+0000 7f814e80c640 1 -- 172.21.15.135:0/1605360176 shutdown_connections 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.220+0000 7f814e80c640 1 --2- 172.21.15.135:0/1605360176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81480d19c0 0x7f81480cfb10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.220+0000 7f814e80c640 1 -- 172.21.15.135:0/1605360176 >> 172.21.15.135:0/1605360176 conn(0x7f814806ce10 msgr2=0x7f814806d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.220+0000 7f814e80c640 1 -- 172.21.15.135:0/1605360176 shutdown_connections 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.220+0000 7f814e80c640 1 -- 172.21.15.135:0/1605360176 wait complete. 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.221+0000 7f814e80c640 1 Processor -- start 2024-04-17T06:09:34.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.221+0000 7f814e80c640 1 -- start start 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f814e80c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81480d19c0 0x7f814814dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f814e80c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81480d2b00 0x7f81481e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f814e80c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f814815eae0 0x7f81481e7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f814e80c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f814815eec0 con 0x7f81480d19c0 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f814e80c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f81480d3aa0 con 0x7f814815eae0 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f814e80c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f81480b9ff0 con 0x7f81480d2b00 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81480d19c0 0x7f814814dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81480d19c0 0x7f814814dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:40046/0 (socket says 172.21.15.135:40046) 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 -- 172.21.15.135:0/3728714502 learned_addr learned my addr 172.21.15.135:0/3728714502 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f81477fe640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81480d2b00 0x7f81481e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f814cda9640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f814815eae0 0x7f81481e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 -- 172.21.15.135:0/3728714502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81480d2b00 msgr2=0x7f81481e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81480d2b00 0x7f81481e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 -- 172.21.15.135:0/3728714502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f814815eae0 msgr2=0x7f81481e7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f814815eae0 0x7f81481e7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 -- 172.21.15.135:0/3728714502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81480ca0d0 con 0x7f81480d19c0 2024-04-17T06:09:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8147fff640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81480d19c0 0x7f814814dc70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f813800c520 tx=0x7f813800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8144ff9640 1 -- 172.21.15.135:0/3728714502 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138017070 con 0x7f81480d19c0 2024-04-17T06:09:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8144ff9640 1 -- 172.21.15.135:0/3728714502 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8138005d00 con 0x7f81480d19c0 2024-04-17T06:09:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f8144ff9640 1 -- 172.21.15.135:0/3728714502 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380072b0 con 0x7f81480d19c0 2024-04-17T06:09:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.222+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81480ca3b0 con 0x7f81480d19c0 2024-04-17T06:09:34.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.223+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81480ca970 con 0x7f81480d19c0 2024-04-17T06:09:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.224+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81480cad50 con 0x7f81480d19c0 2024-04-17T06:09:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.224+0000 7f8144ff9640 1 -- 172.21.15.135:0/3728714502 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f813801a070 con 0x7f81480d19c0 2024-04-17T06:09:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.224+0000 7f8144ff9640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f813406c8f0 0x7f813406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.224+0000 7f8144ff9640 1 -- 172.21.15.135:0/3728714502 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f813808b210 con 0x7f81480d19c0 2024-04-17T06:09:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.224+0000 7f81477fe640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f813406c8f0 0x7f813406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.225+0000 7f81477fe640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f813406c8f0 0x7f813406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f813c036000 tx=0x7f813c03d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.229+0000 7f8144ff9640 1 -- 172.21.15.135:0/3728714502 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81480cad50 con 0x7f81480d19c0 2024-04-17T06:09:34.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.415+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8148068a20 con 0x7f81480d19c0 2024-04-17T06:09:34.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.416+0000 7f8144ff9640 1 -- 172.21.15.135:0/3728714502 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f8138058d40 con 0x7f81480d19c0 2024-04-17T06:09:34.414 INFO:teuthology.orchestra.run.smithi135.stderr:listed 0 entries 2024-04-17T06:09:34.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f813406c8f0 msgr2=0x7f813406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f813406c8f0 0x7f813406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f813c036000 tx=0x7f813c03d040 comp rx=0 tx=0).stop 2024-04-17T06:09:34.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81480d19c0 msgr2=0x7f814814dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81480d19c0 0x7f814814dc70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f813800c520 tx=0x7f813800c9e0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 shutdown_connections 2024-04-17T06:09:34.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f814815eae0 0x7f81481e7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81480d2b00 0x7f81481e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f813406c8f0 0x7f813406edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 --2- 172.21.15.135:0/3728714502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81480d19c0 0x7f814814dc70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 >> 172.21.15.135:0/3728714502 conn(0x7f814806ce10 msgr2=0x7f81480cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:34.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 shutdown_connections 2024-04-17T06:09:34.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.418+0000 7f814e80c640 1 -- 172.21.15.135:0/3728714502 wait complete. 2024-04-17T06:09:34.427 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi135.front.sepia.ceph.com... 2024-04-17T06:09:34.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:34.427 DEBUG:teuthology.orchestra.run.smithi135:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-17T06:09:34.454 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi135.stderr:ceph-fuse[44587]: fuse finished with error 0 and tester_r 0 2024-04-17T06:09:34.456 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T06:09:34.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:09:34.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:34.498 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-17T06:09:34.528 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:34.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.674+0000 7f8962de9640 1 Processor -- start 2024-04-17T06:09:34.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.674+0000 7f8962de9640 1 -- start start 2024-04-17T06:09:34.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.674+0000 7f8962de9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 0x7f895c06e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f8962de9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f895c06d020 0x7f895c070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f8962de9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895c15cea0 con 0x7f895c0d03d0 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f8962de9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f895c0d0940 con 0x7f895c06d020 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f8962de9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f895c06c540 con 0x7f895c15cac0 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f895bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f895c06d020 0x7f895c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f895bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f895c06d020 0x7f895c070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48476/0 (socket says 172.21.15.135:48476) 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f895bfff640 1 -- 172.21.15.135:0/530473747 learned_addr learned my addr 172.21.15.135:0/530473747 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f8960b85640 1 --2- 172.21.15.135:0/530473747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 0x7f895c06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f895bfff640 1 -- 172.21.15.135:0/530473747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 msgr2=0x7f895c06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f895bfff640 1 --2- 172.21.15.135:0/530473747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 0x7f895c06e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f895bfff640 1 -- 172.21.15.135:0/530473747 >> v1:172.21.15.190:6789/0 conn(0x7f895c0d03d0 legacy=0x7f895c0cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.675+0000 7f895bfff640 1 -- 172.21.15.135:0/530473747 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c06c820 con 0x7f895c06d020 2024-04-17T06:09:34.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.676+0000 7f895bfff640 1 --2- 172.21.15.135:0/530473747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c06d020 0x7f895c070370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8948009870 tx=0x7f894802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1529cc8481bd4329 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.676+0000 7f895b7fe640 1 -- 172.21.15.135:0/530473747 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948034070 con 0x7f895c06d020 2024-04-17T06:09:34.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.676+0000 7f895b7fe640 1 -- 172.21.15.135:0/530473747 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8948030840 con 0x7f895c06d020 2024-04-17T06:09:34.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.676+0000 7f895b7fe640 1 -- 172.21.15.135:0/530473747 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948030c10 con 0x7f895c06d020 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.676+0000 7f8962de9640 1 -- 172.21.15.135:0/530473747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c06d020 msgr2=0x7f895c070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.676+0000 7f8962de9640 1 --2- 172.21.15.135:0/530473747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c06d020 0x7f895c070370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8948009870 tx=0x7f894802b980 comp rx=0 tx=0).stop 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.676+0000 7f8962de9640 1 -- 172.21.15.135:0/530473747 shutdown_connections 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.677+0000 7f8962de9640 1 --2- 172.21.15.135:0/530473747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c06d020 0x7f895c070370 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.677+0000 7f8962de9640 1 --2- 172.21.15.135:0/530473747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 0x7f895c06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.677+0000 7f8962de9640 1 -- 172.21.15.135:0/530473747 >> 172.21.15.135:0/530473747 conn(0x7f895c0cd7e0 msgr2=0x7f895c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.677+0000 7f8962de9640 1 -- 172.21.15.135:0/530473747 shutdown_connections 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.677+0000 7f8962de9640 1 -- 172.21.15.135:0/530473747 wait complete. 2024-04-17T06:09:34.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.678+0000 7f8962de9640 1 Processor -- start 2024-04-17T06:09:34.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.678+0000 7f8962de9640 1 -- start start 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8962de9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06d020 0x7f895c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8962de9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c0d03d0 0x7f895c1f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8962de9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 0x7f895c1f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8962de9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f895c06ef10 con 0x7f895c0d03d0 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8962de9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f895c06ed90 con 0x7f895c06d020 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8962de9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f895c0708b0 con 0x7f895c15cac0 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c0d03d0 0x7f895c1f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c0d03d0 0x7f895c1f7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48486/0 (socket says 172.21.15.135:48486) 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 -- 172.21.15.135:0/3640140169 learned_addr learned my addr 172.21.15.135:0/3640140169 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8961386640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 0x7f895c1f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8960b85640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06d020 0x7f895c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 -- 172.21.15.135:0/3640140169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 msgr2=0x7f895c1f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 0x7f895c1f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 -- 172.21.15.135:0/3640140169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06d020 msgr2=0x7f895c1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06d020 0x7f895c1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 -- 172.21.15.135:0/3640140169 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c15ed20 con 0x7f895c0d03d0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f895bfff640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c0d03d0 0x7f895c1f7ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8948009870 tx=0x7f8948004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f89597fa640 1 -- 172.21.15.135:0/3640140169 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948048070 con 0x7f895c0d03d0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f89597fa640 1 -- 172.21.15.135:0/3640140169 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8948030640 con 0x7f895c0d03d0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895c15f000 con 0x7f895c0d03d0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f89597fa640 1 -- 172.21.15.135:0/3640140169 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948034040 con 0x7f895c0d03d0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.679+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895c15f5c0 con 0x7f895c0d03d0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.680+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895c15f9a0 con 0x7f895c0d03d0 2024-04-17T06:09:34.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.681+0000 7f89597fa640 1 -- 172.21.15.135:0/3640140169 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f894804b070 con 0x7f895c0d03d0 2024-04-17T06:09:34.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.681+0000 7f89597fa640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.681+0000 7f89597fa640 1 -- 172.21.15.135:0/3640140169 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f89480add90 con 0x7f895c0d03d0 2024-04-17T06:09:34.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.681+0000 7f8960b85640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.682+0000 7f8960b85640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f894006c8f0 0x7f894006edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8950009a30 tx=0x7f8950008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.686+0000 7f89597fa640 1 -- 172.21.15.135:0/3640140169 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f895c15f9a0 con 0x7f895c0d03d0 2024-04-17T06:09:34.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.863+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f895c074940 con 0x7f895c0d03d0 2024-04-17T06:09:34.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.864+0000 7f89597fa640 1 -- 172.21.15.135:0/3640140169 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f894807be80 con 0x7f895c0d03d0 2024-04-17T06:09:34.862 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:34.862 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:34.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f894006c8f0 msgr2=0x7f894006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f894006c8f0 0x7f894006edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8950009a30 tx=0x7f8950008040 comp rx=0 tx=0).stop 2024-04-17T06:09:34.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c0d03d0 msgr2=0x7f895c1f7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c0d03d0 0x7f895c1f7ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8948009870 tx=0x7f8948004730 comp rx=0 tx=0).stop 2024-04-17T06:09:34.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 shutdown_connections 2024-04-17T06:09:34.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f895c15cac0 0x7f895c1f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f895c0d03d0 0x7f895c1f7ea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 --2- 172.21.15.135:0/3640140169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f895c06d020 0x7f895c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 >> 172.21.15.135:0/3640140169 conn(0x7f895c0cd7e0 msgr2=0x7f895c0affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:34.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 shutdown_connections 2024-04-17T06:09:34.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.866+0000 7f8962de9640 1 -- 172.21.15.135:0/3640140169 wait complete. 2024-04-17T06:09:34.875 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:34.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.979+0000 7f463f30d640 1 Processor -- start 2024-04-17T06:09:34.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.980+0000 7f463f30d640 1 -- start start 2024-04-17T06:09:34.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.980+0000 7f463f30d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46380b07d0 0x7f463806b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.980+0000 7f463f30d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463806c2e0 con 0x7f463806bed0 2024-04-17T06:09:34.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.980+0000 7f463f30d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4638162920 con 0x7f463815ecc0 2024-04-17T06:09:34.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.980+0000 7f463f30d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f46381636e0 con 0x7f46380b07d0 2024-04-17T06:09:34.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f463d8aa640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f463806bed0 0x7f4638073000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37668/0 (socket says 172.21.15.135:37668) 2024-04-17T06:09:34.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f463d8aa640 1 -- 172.21.15.135:0/259294386 learned_addr learned my addr 172.21.15.135:0/259294386 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:34.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f463c8a8640 1 --2- 172.21.15.135:0/259294386 >> v2:172.21.15.190:3300/0 conn(0x7f46380b07d0 0x7f463806b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200380052 0 0) 0x7f4630001110 con 0x7f463806bed0 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4620003520 con 0x7f463806bed0 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1734261291 0 0) 0x7f4620003520 con 0x7f463806bed0 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 >> v2:172.21.15.190:3300/0 conn(0x7f46380b07d0 msgr2=0x7f463806b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 --2- 172.21.15.135:0/259294386 >> v2:172.21.15.190:3300/0 conn(0x7f46380b07d0 0x7f463806b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 >> v1:172.21.15.190:6789/0 conn(0x7f463815ecc0 legacy=0x7f46380ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4638162c00 con 0x7f463806bed0 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f4630001840 con 0x7f463806bed0 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 >> v1:172.21.15.135:6789/0 conn(0x7f463806bed0 legacy=0x7f4638073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f4620003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4620004800 0x7f4620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46200051c0 0x7f462000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46381636e0 con 0x7f46200039b0 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f462000c160 con 0x7f4620004800 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f462000c360 con 0x7f46200051c0 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f463d0a9640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f4620003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f463c8a8640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4620004800 0x7f4620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.981+0000 7f463d8aa640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46200051c0 0x7f462000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f463d0a9640 1 -- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46200051c0 msgr2=0x7f462000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f463d0a9640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46200051c0 0x7f462000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f463d0a9640 1 -- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4620004800 msgr2=0x7f4620004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f463d0a9640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4620004800 0x7f4620004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f463d0a9640 1 -- 172.21.15.135:0/259294386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4620003520 con 0x7f46200039b0 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f463d8aa640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46200051c0 0x7f462000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:34.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f463d8aa640 1 -- 172.21.15.135:0/259294386 reap_dead start 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f463d0a9640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f4620003dd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4628001e50 tx=0x7f4620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f0b856f1ec3171 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628001a70 con 0x7f46200039b0 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4628003b70 con 0x7f46200039b0 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.982+0000 7f4637fff640 1 -- 172.21.15.135:0/259294386 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46280050d0 con 0x7f46200039b0 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.983+0000 7f463f30d640 1 -- 172.21.15.135:0/259294386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 msgr2=0x7f4620003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.983+0000 7f463f30d640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f4620003dd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4628001e50 tx=0x7f4620005830 comp rx=0 tx=0).stop 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.983+0000 7f463f30d640 1 -- 172.21.15.135:0/259294386 shutdown_connections 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.983+0000 7f463f30d640 1 --2- 172.21.15.135:0/259294386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f4620003dd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.983+0000 7f463f30d640 1 -- 172.21.15.135:0/259294386 >> 172.21.15.135:0/259294386 conn(0x7f46380b3320 msgr2=0x7f46380b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.983+0000 7f463f30d640 1 -- 172.21.15.135:0/259294386 shutdown_connections 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.983+0000 7f463f30d640 1 -- 172.21.15.135:0/259294386 wait complete. 2024-04-17T06:09:34.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.984+0000 7f463f30d640 1 Processor -- start 2024-04-17T06:09:34.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.984+0000 7f463f30d640 1 -- start start 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463f30d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f463814f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463f30d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46380ce600 0x7f46380cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463f30d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4638158400 0x7f4638158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463f30d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46381600b0 con 0x7f46200039b0 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463f30d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f46381631c0 con 0x7f4638158400 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463f30d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4638158dc0 con 0x7f46380ce600 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463d0a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f463814f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463d0a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f463814f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48502/0 (socket says 172.21.15.135:48502) 2024-04-17T06:09:34.988 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463d0a9640 1 -- 172.21.15.135:0/1214794293 learned_addr learned my addr 172.21.15.135:0/1214794293 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463d0a9640 1 -- 172.21.15.135:0/1214794293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46380ce600 msgr2=0x7f46380cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463d0a9640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46380ce600 0x7f46380cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463d0a9640 1 -- 172.21.15.135:0/1214794293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4638158400 msgr2=0x7f4638158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463d0a9640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4638158400 0x7f4638158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.985+0000 7f463d0a9640 1 -- 172.21.15.135:0/1214794293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4638157470 con 0x7f46200039b0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.986+0000 7f463d0a9640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f463814f130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4628001f70 tx=0x7f46280014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.986+0000 7f4635ffb640 1 -- 172.21.15.135:0/1214794293 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628003970 con 0x7f46200039b0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.986+0000 7f4635ffb640 1 -- 172.21.15.135:0/1214794293 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4628006b50 con 0x7f46200039b0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.986+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4638157750 con 0x7f46200039b0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.986+0000 7f4635ffb640 1 -- 172.21.15.135:0/1214794293 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628007020 con 0x7f46200039b0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.986+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4638157d10 con 0x7f46200039b0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.987+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4638068a20 con 0x7f46200039b0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.988+0000 7f4635ffb640 1 -- 172.21.15.135:0/1214794293 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46280071c0 con 0x7f46200039b0 2024-04-17T06:09:34.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.988+0000 7f4635ffb640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f462406c920 0x7f462406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:34.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.988+0000 7f4635ffb640 1 -- 172.21.15.135:0/1214794293 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f46280ad4e0 con 0x7f46200039b0 2024-04-17T06:09:34.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.988+0000 7f463c8a8640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f462406c920 0x7f462406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:34.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.989+0000 7f463c8a8640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f462406c920 0x7f462406ede0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f462c006e80 tx=0x7f462c006df0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:34.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:34.992+0000 7f4635ffb640 1 -- 172.21.15.135:0/1214794293 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4628033020 con 0x7f46200039b0 2024-04-17T06:09:35.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.167+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4638070450 con 0x7f46200039b0 2024-04-17T06:09:35.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.169+0000 7f4635ffb640 1 -- 172.21.15.135:0/1214794293 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f462807b5d0 con 0x7f46200039b0 2024-04-17T06:09:35.167 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:35.167 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f462406c920 msgr2=0x7f462406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f462406c920 0x7f462406ede0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f462c006e80 tx=0x7f462c006df0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 msgr2=0x7f463814f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f463814f130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4628001f70 tx=0x7f46280014a0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 shutdown_connections 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4638158400 0x7f4638158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46380ce600 0x7f46380cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f462406c920 0x7f462406ede0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 --2- 172.21.15.135:0/1214794293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46200039b0 0x7f463814f130 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 >> 172.21.15.135:0/1214794293 conn(0x7f46380b3320 msgr2=0x7f4638159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 shutdown_connections 2024-04-17T06:09:35.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.171+0000 7f463f30d640 1 -- 172.21.15.135:0/1214794293 wait complete. 2024-04-17T06:09:35.180 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T06:09:35.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.288+0000 7fbf9982a640 1 Processor -- start 2024-04-17T06:09:35.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.288+0000 7fbf9982a640 1 -- start start 2024-04-17T06:09:35.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf9982a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbf9415c8a0 0x7fbf940b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf9982a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbf940b3d40 0x7fbf940cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf9982a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf9406ba30 con 0x7fbf940cd5e0 2024-04-17T06:09:35.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf9982a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fbf9406c600 con 0x7fbf9415c8a0 2024-04-17T06:09:35.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf9982a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbf9406bcd0 con 0x7fbf940b3d40 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf92ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbf9415c8a0 0x7fbf940b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf92ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbf9415c8a0 0x7fbf940b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48510/0 (socket says 172.21.15.135:48510) 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf937fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fbf940cd5e0 0x7fbf940cf150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37674/0 (socket says 172.21.15.135:37674) 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf92ffd640 1 -- 172.21.15.135:0/1687678463 learned_addr learned my addr 172.21.15.135:0/1687678463 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf927fc640 1 --2- 172.21.15.135:0/1687678463 >> v2:172.21.15.190:3300/0 conn(0x7fbf940b3d40 0x7fbf940cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf92ffd640 1 -- 172.21.15.135:0/1687678463 >> v2:172.21.15.190:3300/0 conn(0x7fbf940b3d40 msgr2=0x7fbf940cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf92ffd640 1 --2- 172.21.15.135:0/1687678463 >> v2:172.21.15.190:3300/0 conn(0x7fbf940b3d40 0x7fbf940cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf92ffd640 1 -- 172.21.15.135:0/1687678463 >> v1:172.21.15.135:6789/0 conn(0x7fbf940cd5e0 legacy=0x7fbf940cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.289+0000 7fbf92ffd640 1 -- 172.21.15.135:0/1687678463 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9406bfb0 con 0x7fbf9415c8a0 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.290+0000 7fbf927fc640 1 --2- 172.21.15.135:0/1687678463 >> v2:172.21.15.190:3300/0 conn(0x7fbf940b3d40 0x7fbf940cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.290+0000 7fbf92ffd640 1 --2- 172.21.15.135:0/1687678463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9415c8a0 0x7fbf940b3800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbf940d2650 tx=0x7fbf8402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec74228fc76db52 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.290+0000 7fbf91ffb640 1 -- 172.21.15.135:0/1687678463 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8402cb20 con 0x7fbf9415c8a0 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.290+0000 7fbf91ffb640 1 -- 172.21.15.135:0/1687678463 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8402ccc0 con 0x7fbf9415c8a0 2024-04-17T06:09:35.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.290+0000 7fbf91ffb640 1 -- 172.21.15.135:0/1687678463 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84034610 con 0x7fbf9415c8a0 2024-04-17T06:09:35.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.291+0000 7fbf9982a640 1 -- 172.21.15.135:0/1687678463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9415c8a0 msgr2=0x7fbf940b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.291+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1687678463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9415c8a0 0x7fbf940b3800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbf940d2650 tx=0x7fbf8402bf80 comp rx=0 tx=0).stop 2024-04-17T06:09:35.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.291+0000 7fbf9982a640 1 -- 172.21.15.135:0/1687678463 shutdown_connections 2024-04-17T06:09:35.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.291+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1687678463 >> v2:172.21.15.190:3300/0 conn(0x7fbf940b3d40 0x7fbf940cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.291+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1687678463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf9415c8a0 0x7fbf940b3800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.291+0000 7fbf9982a640 1 -- 172.21.15.135:0/1687678463 >> 172.21.15.135:0/1687678463 conn(0x7fbf9409a9f0 msgr2=0x7fbf940b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.291+0000 7fbf9982a640 1 -- 172.21.15.135:0/1687678463 shutdown_connections 2024-04-17T06:09:35.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.291+0000 7fbf9982a640 1 -- 172.21.15.135:0/1687678463 wait complete. 2024-04-17T06:09:35.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.292+0000 7fbf9982a640 1 Processor -- start 2024-04-17T06:09:35.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.292+0000 7fbf9982a640 1 -- start start 2024-04-17T06:09:35.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf9982a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf940b3d40 0x7fbf940c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf9982a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf940cd5e0 0x7fbf941f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf9982a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9415c8a0 0x7fbf940caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf9982a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf940cda70 con 0x7fbf940b3d40 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf9982a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9415dc90 con 0x7fbf9415c8a0 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf9982a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbf940ced60 con 0x7fbf940cd5e0 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf940b3d40 0x7fbf940c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf92ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf940b3d40 0x7fbf940c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48518/0 (socket says 172.21.15.135:48518) 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf92ffd640 1 -- 172.21.15.135:0/1137989168 learned_addr learned my addr 172.21.15.135:0/1137989168 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf92ffd640 1 -- 172.21.15.135:0/1137989168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf940cd5e0 msgr2=0x7fbf941f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf92ffd640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf940cd5e0 0x7fbf941f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf92ffd640 1 -- 172.21.15.135:0/1137989168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9415c8a0 msgr2=0x7fbf940caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf92ffd640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9415c8a0 0x7fbf940caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.293+0000 7fbf92ffd640 1 -- 172.21.15.135:0/1137989168 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf941f52f0 con 0x7fbf940b3d40 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.294+0000 7fbf92ffd640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf940b3d40 0x7fbf940c5030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf84000f80 tx=0x7fbf84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.294+0000 7fbf7bfff640 1 -- 172.21.15.135:0/1137989168 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84043020 con 0x7fbf940b3d40 2024-04-17T06:09:35.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.294+0000 7fbf7bfff640 1 -- 172.21.15.135:0/1137989168 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8403e450 con 0x7fbf940b3d40 2024-04-17T06:09:35.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.294+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf940c5630 con 0x7fbf940b3d40 2024-04-17T06:09:35.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.294+0000 7fbf7bfff640 1 -- 172.21.15.135:0/1137989168 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8403e840 con 0x7fbf940b3d40 2024-04-17T06:09:35.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.294+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf940c5bf0 con 0x7fbf940b3d40 2024-04-17T06:09:35.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.295+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf94074940 con 0x7fbf940b3d40 2024-04-17T06:09:35.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.295+0000 7fbf7bfff640 1 -- 172.21.15.135:0/1137989168 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf84046020 con 0x7fbf940b3d40 2024-04-17T06:09:35.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.296+0000 7fbf7bfff640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbf8006c920 0x7fbf8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.296+0000 7fbf927fc640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbf8006c920 0x7fbf8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.296+0000 7fbf7bfff640 1 -- 172.21.15.135:0/1137989168 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fbf840ae6b0 con 0x7fbf940b3d40 2024-04-17T06:09:35.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.296+0000 7fbf927fc640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbf8006c920 0x7fbf8006ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbf88006fd0 tx=0x7fbf88008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.300+0000 7fbf7bfff640 1 -- 172.21.15.135:0/1137989168 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf8404d030 con 0x7fbf940b3d40 2024-04-17T06:09:35.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.519+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbf9406df90 con 0x7fbf940b3d40 2024-04-17T06:09:35.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.690+0000 7fbf7bfff640 1 -- 172.21.15.135:0/1137989168 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fbf941f52f0 con 0x7fbf940b3d40 2024-04-17T06:09:35.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbf8006c920 msgr2=0x7fbf8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbf8006c920 0x7fbf8006ede0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbf88006fd0 tx=0x7fbf88008040 comp rx=0 tx=0).stop 2024-04-17T06:09:35.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf940b3d40 msgr2=0x7fbf940c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf940b3d40 0x7fbf940c5030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf84000f80 tx=0x7fbf84004a40 comp rx=0 tx=0).stop 2024-04-17T06:09:35.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 shutdown_connections 2024-04-17T06:09:35.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbf9415c8a0 0x7fbf940caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbf940cd5e0 0x7fbf941f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbf8006c920 0x7fbf8006ede0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 --2- 172.21.15.135:0/1137989168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf940b3d40 0x7fbf940c5030 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 >> 172.21.15.135:0/1137989168 conn(0x7fbf9409a9f0 msgr2=0x7fbf940ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 shutdown_connections 2024-04-17T06:09:35.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.693+0000 7fbf9982a640 1 -- 172.21.15.135:0/1137989168 wait complete. 2024-04-17T06:09:35.703 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-17T06:09:35.703 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T06:09:35.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.810+0000 7f48d7c60640 1 Processor -- start 2024-04-17T06:09:35.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.810+0000 7f48d7c60640 1 -- start start 2024-04-17T06:09:35.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d7c60640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f48d0158a50 0x7f48d015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d7c60640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158e30 0x7f48d006f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d7c60640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48d00b3590 0x7f48d00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d7c60640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f48d006fb50 con 0x7f48d00b3590 2024-04-17T06:09:35.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d7c60640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f48d015b840 con 0x7f48d0158a50 2024-04-17T06:09:35.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d7c60640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f48d015fb80 con 0x7f48d0158e30 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d61fd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48d00b3590 0x7f48d00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d61fd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f48d00b3590 0x7f48d00b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48534/0 (socket says 172.21.15.135:48534) 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d61fd640 1 -- 172.21.15.135:0/870849486 learned_addr learned my addr 172.21.15.135:0/870849486 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d51fb640 1 --2- 172.21.15.135:0/870849486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158e30 0x7f48d006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d59fc640 1 --2- 172.21.15.135:0/870849486 >> v2:172.21.15.190:3300/0 conn(0x7f48d0158a50 0x7f48d015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d61fd640 1 -- 172.21.15.135:0/870849486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158e30 msgr2=0x7f48d006f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d61fd640 1 --2- 172.21.15.135:0/870849486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158e30 0x7f48d006f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d61fd640 1 -- 172.21.15.135:0/870849486 >> v2:172.21.15.190:3300/0 conn(0x7f48d0158a50 msgr2=0x7f48d015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d61fd640 1 --2- 172.21.15.135:0/870849486 >> v2:172.21.15.190:3300/0 conn(0x7f48d0158a50 0x7f48d015ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.811+0000 7f48d61fd640 1 -- 172.21.15.135:0/870849486 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d015f2e0 con 0x7f48d00b3590 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.812+0000 7f48d61fd640 1 --2- 172.21.15.135:0/870849486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d00b3a40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f48d006ce10 tx=0x7f48c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d203c70f23f79f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.812+0000 7f48d49fa640 1 -- 172.21.15.135:0/870849486 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8035020 con 0x7f48d00b3590 2024-04-17T06:09:35.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.812+0000 7f48d49fa640 1 -- 172.21.15.135:0/870849486 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c8031640 con 0x7f48d00b3590 2024-04-17T06:09:35.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.812+0000 7f48d49fa640 1 -- 172.21.15.135:0/870849486 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c802d040 con 0x7f48d00b3590 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 -- 172.21.15.135:0/870849486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 msgr2=0x7f48d00b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 --2- 172.21.15.135:0/870849486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d00b3a40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f48d006ce10 tx=0x7f48c802b8f0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 -- 172.21.15.135:0/870849486 shutdown_connections 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 --2- 172.21.15.135:0/870849486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d00b3a40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 --2- 172.21.15.135:0/870849486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158e30 0x7f48d006f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 --2- 172.21.15.135:0/870849486 >> v2:172.21.15.190:3300/0 conn(0x7f48d0158a50 0x7f48d015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 -- 172.21.15.135:0/870849486 >> 172.21.15.135:0/870849486 conn(0x7f48d00d0520 msgr2=0x7f48d00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 -- 172.21.15.135:0/870849486 shutdown_connections 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.813+0000 7f48d7c60640 1 -- 172.21.15.135:0/870849486 wait complete. 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.814+0000 7f48d7c60640 1 Processor -- start 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.814+0000 7f48d7c60640 1 -- start start 2024-04-17T06:09:35.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d7c60640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d01574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d7c60640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158a50 0x7f48d0156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d7c60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d0158e30 0x7f48d0071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d7c60640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48d015bcd0 con 0x7f48d00b3590 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d7c60640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f48d015f8a0 con 0x7f48d0158e30 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d7c60640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f48d015be50 con 0x7f48d0158a50 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d01574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d01574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48538/0 (socket says 172.21.15.135:48538) 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 -- 172.21.15.135:0/1973058895 learned_addr learned my addr 172.21.15.135:0/1973058895 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d61fd640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d0158e30 0x7f48d0071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d51fb640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158a50 0x7f48d0156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 -- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158a50 msgr2=0x7f48d0156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158a50 0x7f48d0156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 -- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d0158e30 msgr2=0x7f48d0071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:35.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d0158e30 0x7f48d0071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 -- 172.21.15.135:0/1973058895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d0154700 con 0x7f48d00b3590 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d51fb640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158a50 0x7f48d0156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d59fc640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d01574f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00eed0 tx=0x7f48cc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48c67fc640 1 -- 172.21.15.135:0/1973058895 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc018020 con 0x7f48d00b3590 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48c67fc640 1 -- 172.21.15.135:0/1973058895 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48cc014400 con 0x7f48d00b3590 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d01549e0 con 0x7f48d00b3590 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48c67fc640 1 -- 172.21.15.135:0/1973058895 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc0147d0 con 0x7f48d00b3590 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.815+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d0154fa0 con 0x7f48d00b3590 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.816+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d00cd7f0 con 0x7f48d00b3590 2024-04-17T06:09:35.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.817+0000 7f48c67fc640 1 -- 172.21.15.135:0/1973058895 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48cc01b020 con 0x7f48d00b3590 2024-04-17T06:09:35.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.817+0000 7f48c67fc640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f48b406c920 0x7f48b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:35.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.817+0000 7f48c67fc640 1 -- 172.21.15.135:0/1973058895 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f48cc08c610 con 0x7f48d00b3590 2024-04-17T06:09:35.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.817+0000 7f48d51fb640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f48b406c920 0x7f48b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:35.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.818+0000 7f48d51fb640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f48b406c920 0x7f48b406ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48d00aea90 tx=0x7f48c0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:35.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:35.822+0000 7f48c67fc640 1 -- 172.21.15.135:0/1973058895 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48d00cd7f0 con 0x7f48d00b3590 2024-04-17T06:09:36.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.037+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f48d015ec50 con 0x7f48d00b3590 2024-04-17T06:09:36.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.703+0000 7f48c67fc640 1 -- 172.21.15.135:0/1973058895 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f48cc022030 con 0x7f48d00b3590 2024-04-17T06:09:36.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.705+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f48b406c920 msgr2=0x7f48b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:36.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.705+0000 7f48d7c60640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f48b406c920 0x7f48b406ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48d00aea90 tx=0x7f48c0008040 comp rx=0 tx=0).stop 2024-04-17T06:09:36.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.705+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 msgr2=0x7f48d01574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:36.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.705+0000 7f48d7c60640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d01574f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00eed0 tx=0x7f48cc00b480 comp rx=0 tx=0).stop 2024-04-17T06:09:36.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.706+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 shutdown_connections 2024-04-17T06:09:36.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.706+0000 7f48d7c60640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48d0158e30 0x7f48d0071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.706+0000 7f48d7c60640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48d0158a50 0x7f48d0156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.706+0000 7f48d7c60640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f48b406c920 0x7f48b406ede0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.706+0000 7f48d7c60640 1 --2- 172.21.15.135:0/1973058895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48d00b3590 0x7f48d01574f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.706+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 >> 172.21.15.135:0/1973058895 conn(0x7f48d00d0520 msgr2=0x7f48d00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:36.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.706+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 shutdown_connections 2024-04-17T06:09:36.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.706+0000 7f48d7c60640 1 -- 172.21.15.135:0/1973058895 wait complete. 2024-04-17T06:09:36.716 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T06:09:36.716 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:09:36.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.824+0000 7f1b5649a640 1 Processor -- start 2024-04-17T06:09:36.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.824+0000 7f1b5649a640 1 -- start start 2024-04-17T06:09:36.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b5649a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1b5015f800 0x7f1b50070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:36.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b5649a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b500d0f20 0x7f1b500cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:36.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b5649a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5006bb30 con 0x7f1b50070c90 2024-04-17T06:09:36.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b5649a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5006c8f0 con 0x7f1b5015f800 2024-04-17T06:09:36.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b5649a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5006f070 con 0x7f1b500d0f20 2024-04-17T06:09:36.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b4ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1b5015f800 0x7f1b50070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:36.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b4ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1b5015f800 0x7f1b50070750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48542/0 (socket says 172.21.15.135:48542) 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b4ffff640 1 -- 172.21.15.135:0/3916168588 learned_addr learned my addr 172.21.15.135:0/3916168588 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b54a37640 1 --2- 172.21.15.135:0/3916168588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b500d0f20 0x7f1b500cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b4ffff640 1 -- 172.21.15.135:0/3916168588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b500d0f20 msgr2=0x7f1b500cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b4ffff640 1 --2- 172.21.15.135:0/3916168588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b500d0f20 0x7f1b500cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b4ffff640 1 -- 172.21.15.135:0/3916168588 >> v1:172.21.15.190:6789/0 conn(0x7f1b50070c90 legacy=0x7f1b500d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.825+0000 7f1b4ffff640 1 -- 172.21.15.135:0/3916168588 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5006be10 con 0x7f1b5015f800 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.826+0000 7f1b54a37640 1 --2- 172.21.15.135:0/3916168588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b500d0f20 0x7f1b500cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.826+0000 7f1b4ffff640 1 --2- 172.21.15.135:0/3916168588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b5015f800 0x7f1b50070750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b400098a0 tx=0x7f1b4002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c07473233a2586 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.826+0000 7f1b4effd640 1 -- 172.21.15.135:0/3916168588 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40034070 con 0x7f1b5015f800 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.826+0000 7f1b4effd640 1 -- 172.21.15.135:0/3916168588 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b40030840 con 0x7f1b5015f800 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.826+0000 7f1b4effd640 1 -- 172.21.15.135:0/3916168588 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40030c10 con 0x7f1b5015f800 2024-04-17T06:09:36.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.827+0000 7f1b5649a640 1 -- 172.21.15.135:0/3916168588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b5015f800 msgr2=0x7f1b50070750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:36.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.827+0000 7f1b5649a640 1 --2- 172.21.15.135:0/3916168588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b5015f800 0x7f1b50070750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1b400098a0 tx=0x7f1b4002b980 comp rx=0 tx=0).stop 2024-04-17T06:09:36.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.827+0000 7f1b5649a640 1 -- 172.21.15.135:0/3916168588 shutdown_connections 2024-04-17T06:09:36.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.827+0000 7f1b5649a640 1 --2- 172.21.15.135:0/3916168588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b500d0f20 0x7f1b500cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.827+0000 7f1b5649a640 1 --2- 172.21.15.135:0/3916168588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b5015f800 0x7f1b50070750 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.827+0000 7f1b5649a640 1 -- 172.21.15.135:0/3916168588 >> 172.21.15.135:0/3916168588 conn(0x7f1b5009a9f0 msgr2=0x7f1b500b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:36.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.827+0000 7f1b5649a640 1 -- 172.21.15.135:0/3916168588 shutdown_connections 2024-04-17T06:09:36.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.827+0000 7f1b5649a640 1 -- 172.21.15.135:0/3916168588 wait complete. 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.828+0000 7f1b5649a640 1 Processor -- start 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.828+0000 7f1b5649a640 1 -- start start 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b5649a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b50070c90 0x7f1b501f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b5649a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b500d0f20 0x7f1b500c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b5649a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b5015f800 0x7f1b500af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b5649a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5015fbe0 con 0x7f1b50070c90 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b5649a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5006c460 con 0x7f1b500d0f20 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b5649a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5015fd60 con 0x7f1b5015f800 2024-04-17T06:09:36.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b50070c90 0x7f1b501f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b50070c90 0x7f1b501f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48548/0 (socket says 172.21.15.135:48548) 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4ffff640 1 -- 172.21.15.135:0/417895359 learned_addr learned my addr 172.21.15.135:0/417895359 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b54a37640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b5015f800 0x7f1b500af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4f7fe640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b500d0f20 0x7f1b500c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4ffff640 1 -- 172.21.15.135:0/417895359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b5015f800 msgr2=0x7f1b500af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4ffff640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b5015f800 0x7f1b500af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4ffff640 1 -- 172.21.15.135:0/417895359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b500d0f20 msgr2=0x7f1b500c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4ffff640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b500d0f20 0x7f1b500c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.829+0000 7f1b4ffff640 1 -- 172.21.15.135:0/417895359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b501f5960 con 0x7f1b50070c90 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.830+0000 7f1b4ffff640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b50070c90 0x7f1b501f5420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b40000c00 tx=0x7f1b40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.830+0000 7f1b4cff9640 1 -- 172.21.15.135:0/417895359 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40048070 con 0x7f1b50070c90 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.830+0000 7f1b4cff9640 1 -- 172.21.15.135:0/417895359 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b40030640 con 0x7f1b50070c90 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.830+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b500b6070 con 0x7f1b50070c90 2024-04-17T06:09:36.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.830+0000 7f1b4cff9640 1 -- 172.21.15.135:0/417895359 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40034040 con 0x7f1b50070c90 2024-04-17T06:09:36.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.830+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b500b6630 con 0x7f1b50070c90 2024-04-17T06:09:36.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.831+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b50074940 con 0x7f1b50070c90 2024-04-17T06:09:36.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.831+0000 7f1b4cff9640 1 -- 172.21.15.135:0/417895359 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b4004b070 con 0x7f1b50070c90 2024-04-17T06:09:36.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.832+0000 7f1b4cff9640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1b3c06c8f0 0x7f1b3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:36.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.832+0000 7f1b4cff9640 1 -- 172.21.15.135:0/417895359 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1b400adf90 con 0x7f1b50070c90 2024-04-17T06:09:36.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.832+0000 7f1b4f7fe640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1b3c06c8f0 0x7f1b3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:36.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.832+0000 7f1b4f7fe640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1b3c06c8f0 0x7f1b3c06edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b44006fd0 tx=0x7f1b44008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:36.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:36.836+0000 7f1b4cff9640 1 -- 172.21.15.135:0/417895359 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b40030810 con 0x7f1b50070c90 2024-04-17T06:09:37.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.046+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b50073be0 con 0x7f1b50070c90 2024-04-17T06:09:37.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.046+0000 7f1b4cff9640 1 -- 172.21.15.135:0/417895359 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f1b4007c230 con 0x7f1b50070c90 2024-04-17T06:09:37.045 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:37.045 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:36.038772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:09:37.045 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 9 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1b3c06c8f0 msgr2=0x7f1b3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1b3c06c8f0 0x7f1b3c06edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b44006fd0 tx=0x7f1b44008040 comp rx=0 tx=0).stop 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b50070c90 msgr2=0x7f1b501f5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b50070c90 0x7f1b501f5420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b40000c00 tx=0x7f1b40004730 comp rx=0 tx=0).stop 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 shutdown_connections 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b5015f800 0x7f1b500af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b500d0f20 0x7f1b500c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1b3c06c8f0 0x7f1b3c06edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 --2- 172.21.15.135:0/417895359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b50070c90 0x7f1b501f5420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 >> 172.21.15.135:0/417895359 conn(0x7f1b5009a9f0 msgr2=0x7f1b5015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:37.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 shutdown_connections 2024-04-17T06:09:37.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.048+0000 7f1b5649a640 1 -- 172.21.15.135:0/417895359 wait complete. 2024-04-17T06:09:37.058 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:37.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.162+0000 7f101ed72640 1 Processor -- start 2024-04-17T06:09:37.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.162+0000 7f101ed72640 1 -- start start 2024-04-17T06:09:37.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101ed72640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f101815bb00 0x7f10180b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101ed72640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10180afdd0 0x7f10180ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101ed72640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10180b3a50 con 0x7f10180b3590 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101ed72640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f10180b4660 con 0x7f10180afdd0 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101ed72640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f10180b3d30 con 0x7f101815bb00 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101cb0e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f101815bb00 0x7f10180b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101d30f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10180afdd0 0x7f10180ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f100ffff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f10180b3590 0x7f10180af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39290/0 (socket says 172.21.15.135:39290) 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f100ffff640 1 -- 172.21.15.135:0/2989475046 learned_addr learned my addr 172.21.15.135:0/2989475046 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f100f7fe640 1 -- 172.21.15.135:0/2989475046 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4280144242 0 0) 0x7f10180b3a50 con 0x7f10180b3590 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f100f7fe640 1 -- 172.21.15.135:0/2989475046 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1000003520 con 0x7f10180b3590 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101cb0e640 1 -- 172.21.15.135:0/2989475046 >> v2:172.21.15.190:3300/0 conn(0x7f10180afdd0 msgr2=0x7f10180ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101cb0e640 1 --2- 172.21.15.135:0/2989475046 >> v2:172.21.15.190:3300/0 conn(0x7f10180afdd0 0x7f10180ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101cb0e640 1 -- 172.21.15.135:0/2989475046 >> v1:172.21.15.190:6789/0 conn(0x7f10180b3590 legacy=0x7f10180af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101cb0e640 1 -- 172.21.15.135:0/2989475046 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10180b4010 con 0x7f101815bb00 2024-04-17T06:09:37.163 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.163+0000 7f101d30f640 1 --2- 172.21.15.135:0/2989475046 >> v2:172.21.15.190:3300/0 conn(0x7f10180afdd0 0x7f10180ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:37.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.164+0000 7f101cb0e640 1 --2- 172.21.15.135:0/2989475046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f101815bb00 0x7f10180b5150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f10140097a0 tx=0x7f101402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1038e6a8ce7b7a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.164+0000 7f100f7fe640 1 -- 172.21.15.135:0/2989475046 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014035020 con 0x7f101815bb00 2024-04-17T06:09:37.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.164+0000 7f100f7fe640 1 -- 172.21.15.135:0/2989475046 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1014031830 con 0x7f101815bb00 2024-04-17T06:09:37.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.164+0000 7f100f7fe640 1 -- 172.21.15.135:0/2989475046 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014031be0 con 0x7f101815bb00 2024-04-17T06:09:37.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.165+0000 7f101ed72640 1 -- 172.21.15.135:0/2989475046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f101815bb00 msgr2=0x7f10180b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.165+0000 7f101ed72640 1 --2- 172.21.15.135:0/2989475046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f101815bb00 0x7f10180b5150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f10140097a0 tx=0x7f101402b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.165+0000 7f101ed72640 1 -- 172.21.15.135:0/2989475046 shutdown_connections 2024-04-17T06:09:37.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.165+0000 7f101ed72640 1 --2- 172.21.15.135:0/2989475046 >> v2:172.21.15.190:3300/0 conn(0x7f10180afdd0 0x7f10180ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.165+0000 7f101ed72640 1 --2- 172.21.15.135:0/2989475046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f101815bb00 0x7f10180b5150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.165+0000 7f101ed72640 1 -- 172.21.15.135:0/2989475046 >> 172.21.15.135:0/2989475046 conn(0x7f10180d0520 msgr2=0x7f10180d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:37.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.165+0000 7f101ed72640 1 -- 172.21.15.135:0/2989475046 shutdown_connections 2024-04-17T06:09:37.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.165+0000 7f101ed72640 1 -- 172.21.15.135:0/2989475046 wait complete. 2024-04-17T06:09:37.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.166+0000 7f101ed72640 1 Processor -- start 2024-04-17T06:09:37.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.166+0000 7f101ed72640 1 -- start start 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101ed72640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180afdd0 0x7f101815d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101ed72640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180b3590 0x7f10181513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101ed72640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101815bb00 0x7f10181e8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101ed72640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10180b5810 con 0x7f101815bb00 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101ed72640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f10180b5690 con 0x7f10180b3590 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101ed72640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f10180cec60 con 0x7f10180afdd0 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101d30f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101815bb00 0x7f10181e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101d30f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101815bb00 0x7f10181e8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48550/0 (socket says 172.21.15.135:48550) 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101d30f640 1 -- 172.21.15.135:0/4279423700 learned_addr learned my addr 172.21.15.135:0/4279423700 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f101cb0e640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180afdd0 0x7f101815d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.167+0000 7f100ffff640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180b3590 0x7f10181513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f101d30f640 1 -- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180afdd0 msgr2=0x7f101815d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f101d30f640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180afdd0 0x7f101815d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f101d30f640 1 -- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180b3590 msgr2=0x7f10181513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f101d30f640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180b3590 0x7f10181513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f101d30f640 1 -- 172.21.15.135:0/4279423700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1018151920 con 0x7f101815bb00 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f100ffff640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180b3590 0x7f10181513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f101d30f640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101815bb00 0x7f10181e8620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1010009eb0 tx=0x7f101000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f100d7fa640 1 -- 172.21.15.135:0/4279423700 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010018020 con 0x7f101815bb00 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f100d7fa640 1 -- 172.21.15.135:0/4279423700 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1010014480 con 0x7f101815bb00 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101815e610 con 0x7f101815bb00 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f100d7fa640 1 -- 172.21.15.135:0/4279423700 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1010014850 con 0x7f101815bb00 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.168+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101815ebd0 con 0x7f101815bb00 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.169+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1018074940 con 0x7f101815bb00 2024-04-17T06:09:37.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.169+0000 7f100d7fa640 1 -- 172.21.15.135:0/4279423700 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1010005d10 con 0x7f101815bb00 2024-04-17T06:09:37.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.170+0000 7f100d7fa640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.170+0000 7f100d7fa640 1 -- 172.21.15.135:0/4279423700 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f101008cc60 con 0x7f101815bb00 2024-04-17T06:09:37.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.170+0000 7f101cb0e640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.170+0000 7f101cb0e640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f100406c8f0 0x7f100406edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1014036000 tx=0x7f1014008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.174+0000 7f100d7fa640 1 -- 172.21.15.135:0/4279423700 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1010022030 con 0x7f101815bb00 2024-04-17T06:09:37.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.349+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10180b0740 con 0x7f101815bb00 2024-04-17T06:09:37.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.350+0000 7f100d7fa640 1 -- 172.21.15.135:0/4279423700 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f1010010040 con 0x7f101815bb00 2024-04-17T06:09:37.349 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:37.349 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:37.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f100406c8f0 msgr2=0x7f100406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f100406c8f0 0x7f100406edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1014036000 tx=0x7f1014008030 comp rx=0 tx=0).stop 2024-04-17T06:09:37.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101815bb00 msgr2=0x7f10181e8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101815bb00 0x7f10181e8620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1010009eb0 tx=0x7f101000c490 comp rx=0 tx=0).stop 2024-04-17T06:09:37.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 shutdown_connections 2024-04-17T06:09:37.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f100406c8f0 0x7f100406edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f101815bb00 0x7f10181e8620 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10180b3590 0x7f10181513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 --2- 172.21.15.135:0/4279423700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10180afdd0 0x7f101815d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 >> 172.21.15.135:0/4279423700 conn(0x7f10180d0520 msgr2=0x7f1018151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:37.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 shutdown_connections 2024-04-17T06:09:37.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.352+0000 7f101ed72640 1 -- 172.21.15.135:0/4279423700 wait complete. 2024-04-17T06:09:37.361 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T06:09:37.362 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T06:09:37.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.467+0000 7f50dac88640 1 Processor -- start 2024-04-17T06:09:37.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.467+0000 7f50dac88640 1 -- start start 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.467+0000 7f50dac88640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f50d415c8a0 0x7f50d40b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.467+0000 7f50dac88640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d406ba30 con 0x7f50d40b3d40 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.467+0000 7f50dac88640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d406bcd0 con 0x7f50d40cd7b0 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.467+0000 7f50dac88640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f50d406c600 con 0x7f50d415c8a0 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.468+0000 7f50d8a24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f50d415c8a0 0x7f50d40b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.468+0000 7f50d8a24640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f50d415c8a0 0x7f50d40b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48566/0 (socket says 172.21.15.135:48566) 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.468+0000 7f50d8a24640 1 -- 172.21.15.135:0/4095024451 learned_addr learned my addr 172.21.15.135:0/4095024451 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.468+0000 7f50d3fff640 1 --1- 172.21.15.135:0/4095024451 >> v1:172.21.15.135:6789/0 conn(0x7f50d40b3d40 0x7f50d40cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37676/0 (socket says 172.21.15.135:37676) 2024-04-17T06:09:37.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.468+0000 7f50d8a24640 1 -- 172.21.15.135:0/4095024451 >> v1:172.21.15.190:6789/0 conn(0x7f50d40cd7b0 legacy=0x7f50d415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.468+0000 7f50d8a24640 1 -- 172.21.15.135:0/4095024451 >> v1:172.21.15.135:6789/0 conn(0x7f50d40b3d40 legacy=0x7f50d40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.468+0000 7f50d8a24640 1 -- 172.21.15.135:0/4095024451 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d406bfb0 con 0x7f50d415c8a0 2024-04-17T06:09:37.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.468+0000 7f50d37fe640 1 -- 172.21.15.135:0/4095024451 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175831207 0 0) 0x7f50d406ba30 con 0x7f50d40b3d40 2024-04-17T06:09:37.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50d8a24640 1 --2- 172.21.15.135:0/4095024451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d415c8a0 0x7f50d40b3800 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7a5478d497bb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50d37fe640 1 -- 172.21.15.135:0/4095024451 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8034070 con 0x7f50d415c8a0 2024-04-17T06:09:37.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50d37fe640 1 -- 172.21.15.135:0/4095024451 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c8030860 con 0x7f50d415c8a0 2024-04-17T06:09:37.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50d37fe640 1 -- 172.21.15.135:0/4095024451 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8030c30 con 0x7f50d415c8a0 2024-04-17T06:09:37.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50dac88640 1 -- 172.21.15.135:0/4095024451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d415c8a0 msgr2=0x7f50d40b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50dac88640 1 --2- 172.21.15.135:0/4095024451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d415c8a0 0x7f50d40b3800 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009870 tx=0x7f50c802b9a0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50dac88640 1 -- 172.21.15.135:0/4095024451 shutdown_connections 2024-04-17T06:09:37.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50dac88640 1 --2- 172.21.15.135:0/4095024451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d415c8a0 0x7f50d40b3800 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50dac88640 1 -- 172.21.15.135:0/4095024451 >> 172.21.15.135:0/4095024451 conn(0x7f50d409a9f0 msgr2=0x7f50d40b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:37.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50dac88640 1 -- 172.21.15.135:0/4095024451 shutdown_connections 2024-04-17T06:09:37.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.469+0000 7f50dac88640 1 -- 172.21.15.135:0/4095024451 wait complete. 2024-04-17T06:09:37.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.470+0000 7f50dac88640 1 Processor -- start 2024-04-17T06:09:37.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.471+0000 7f50dac88640 1 -- start start 2024-04-17T06:09:37.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.471+0000 7f50dac88640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d40b3d40 0x7f50d41f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.471+0000 7f50dac88640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d40cd7b0 0x7f50d40b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.471+0000 7f50dac88640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d415c8a0 0x7f50d415e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.471+0000 7f50dac88640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f50d40d0370 con 0x7f50d40cd7b0 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.471+0000 7f50dac88640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f50d40ced60 con 0x7f50d40b3d40 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.471+0000 7f50dac88640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f50d40d04f0 con 0x7f50d415c8a0 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d40cd7b0 0x7f50d40b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d40cd7b0 0x7f50d40b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48574/0 (socket says 172.21.15.135:48574) 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 -- 172.21.15.135:0/3542384829 learned_addr learned my addr 172.21.15.135:0/3542384829 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d9225640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d415c8a0 0x7f50d415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d8a24640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d40b3d40 0x7f50d41f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 -- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d415c8a0 msgr2=0x7f50d415e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d415c8a0 0x7f50d415e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 -- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d40b3d40 msgr2=0x7f50d41f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d40b3d40 0x7f50d41f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 -- 172.21.15.135:0/3542384829 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d40c9be0 con 0x7f50d40cd7b0 2024-04-17T06:09:37.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d9225640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d415c8a0 0x7f50d415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d8a24640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d40b3d40 0x7f50d41f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d3fff640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d40cd7b0 0x7f50d40b4d20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c460 tx=0x7f50c000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d17fa640 1 -- 172.21.15.135:0/3542384829 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0018070 con 0x7f50d40cd7b0 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d17fa640 1 -- 172.21.15.135:0/3542384829 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c0007480 con 0x7f50d40cd7b0 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d40c9ec0 con 0x7f50d40cd7b0 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50d17fa640 1 -- 172.21.15.135:0/3542384829 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0004a00 con 0x7f50d40cd7b0 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.472+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d40b5260 con 0x7f50d40cd7b0 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.473+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d4074940 con 0x7f50d40cd7b0 2024-04-17T06:09:37.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.474+0000 7f50d17fa640 1 -- 172.21.15.135:0/3542384829 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50c001b070 con 0x7f50d40cd7b0 2024-04-17T06:09:37.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.474+0000 7f50d17fa640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f50b806c8f0 0x7f50b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.474+0000 7f50d17fa640 1 -- 172.21.15.135:0/3542384829 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f50c008afe0 con 0x7f50d40cd7b0 2024-04-17T06:09:37.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.474+0000 7f50d8a24640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f50b806c8f0 0x7f50b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.474+0000 7f50d8a24640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f50b806c8f0 0x7f50b806edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50c8036000 tx=0x7f50c803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.478+0000 7f50d17fa640 1 -- 172.21.15.135:0/3542384829 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50c0059160 con 0x7f50d40cd7b0 2024-04-17T06:09:37.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.652+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 --> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50d41f3140 con 0x7f50b806c8f0 2024-04-17T06:09:37.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.660+0000 7f50d17fa640 1 -- 172.21.15.135:0/3542384829 <== mgr.4113 v2:172.21.15.135:6832/3190814219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f50c80309a0 con 0x7f50b806c8f0 2024-04-17T06:09:37.662 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:37.664 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-17T06:09:37.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f50b806c8f0 msgr2=0x7f50b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f50b806c8f0 0x7f50b806edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50c8036000 tx=0x7f50c803b040 comp rx=0 tx=0).stop 2024-04-17T06:09:37.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d40cd7b0 msgr2=0x7f50d40b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d40cd7b0 0x7f50d40b4d20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c460 tx=0x7f50c000c920 comp rx=0 tx=0).stop 2024-04-17T06:09:37.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 shutdown_connections 2024-04-17T06:09:37.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50d415c8a0 0x7f50d415e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f50b806c8f0 0x7f50b806edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50d40cd7b0 0x7f50d40b4d20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 --2- 172.21.15.135:0/3542384829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d40b3d40 0x7f50d41f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 >> 172.21.15.135:0/3542384829 conn(0x7f50d409a9f0 msgr2=0x7f50d40b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:37.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 shutdown_connections 2024-04-17T06:09:37.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.664+0000 7f50dac88640 1 -- 172.21.15.135:0/3542384829 wait complete. 2024-04-17T06:09:37.673 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-04-17T06:09:35.940966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.281588"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012356+0000","last_change":"2024-04-17T06:09:19.632439+0000","last_active":"2024-04-17T06:09:26.012356+0000","last_peered":"2024-04-17T06:09:26.012356+0000","last_clean":"2024-04-17T06:09:26.012356+0000","last_became_active":"2024-04-17T06:09:19.632170+0000","last_became_peered":"2024-04-17T06:09:19.632170+0000","last_unstale":"2024-04-17T06:09:26.012356+0000","last_undegraded":"2024-04-17T06:09:26.012356+0000","last_fullsized":"2024-04-17T06:09:26.012356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:58.547080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011716+0000","last_change":"2024-04-17T06:09:20.647290+0000","last_active":"2024-04-17T06:09:26.011716+0000","last_peered":"2024-04-17T06:09:26.011716+0000","last_clean":"2024-04-17T06:09:26.011716+0000","last_became_active":"2024-04-17T06:09:20.646950+0000","last_became_peered":"2024-04-17T06:09:20.646950+0000","last_unstale":"2024-04-17T06:09:26.011716+0000","last_undegraded":"2024-04-17T06:09:26.011716+0000","last_fullsized":"2024-04-17T06:09:26.011716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011791+0000","last_change":"2024-04-17T06:09:24.992961+0000","last_active":"2024-04-17T06:09:26.011791+0000","last_peered":"2024-04-17T06:09:26.011791+0000","last_clean":"2024-04-17T06:09:26.011791+0000","last_became_active":"2024-04-17T06:09:24.992540+0000","last_became_peered":"2024-04-17T06:09:24.992540+0000","last_unstale":"2024-04-17T06:09:26.011791+0000","last_undegraded":"2024-04-17T06:09:26.011791+0000","last_fullsized":"2024-04-17T06:09:26.011791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:42.943499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011930+0000","last_change":"2024-04-17T06:09:19.632406+0000","last_active":"2024-04-17T06:09:26.011930+0000","last_peered":"2024-04-17T06:09:26.011930+0000","last_clean":"2024-04-17T06:09:26.011930+0000","last_became_active":"2024-04-17T06:09:19.631990+0000","last_became_peered":"2024-04-17T06:09:19.631990+0000","last_unstale":"2024-04-17T06:09:26.011930+0000","last_undegraded":"2024-04-17T06:09:26.011930+0000","last_fullsized":"2024-04-17T06:09:26.011930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:35.753743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012065+0000","last_change":"2024-04-17T06:09:20.631799+0000","last_active":"2024-04-17T06:09:26.012065+0000","last_peered":"2024-04-17T06:09:26.012065+0000","last_clean":"2024-04-17T06:09:26.012065+0000","last_became_active":"2024-04-17T06:09:20.631551+0000","last_became_peered":"2024-04-17T06:09:20.631551+0000","last_unstale":"2024-04-17T06:09:26.012065+0000","last_undegraded":"2024-04-17T06:09:26.012065+0000","last_fullsized":"2024-04-17T06:09:26.012065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011413+0000","last_change":"2024-04-17T06:09:24.681573+0000","last_active":"2024-04-17T06:09:26.011413+0000","last_peered":"2024-04-17T06:09:26.011413+0000","last_clean":"2024-04-17T06:09:26.011413+0000","last_became_active":"2024-04-17T06:09:24.681194+0000","last_became_peered":"2024-04-17T06:09:24.681194+0000","last_unstale":"2024-04-17T06:09:26.011413+0000","last_undegraded":"2024-04-17T06:09:26.011413+0000","last_fullsized":"2024-04-17T06:09:26.011413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010848+0000","last_change":"2024-04-17T06:09:19.626563+0000","last_active":"2024-04-17T06:09:26.010848+0000","last_peered":"2024-04-17T06:09:26.010848+0000","last_clean":"2024-04-17T06:09:26.010848+0000","last_became_active":"2024-04-17T06:09:19.626169+0000","last_became_peered":"2024-04-17T06:09:19.626169+0000","last_unstale":"2024-04-17T06:09:26.010848+0000","last_undegraded":"2024-04-17T06:09:26.010848+0000","last_fullsized":"2024-04-17T06:09:26.010848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:07.389883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660478+0000","last_change":"2024-04-17T06:09:20.629675+0000","last_active":"2024-04-17T06:09:25.660478+0000","last_peered":"2024-04-17T06:09:25.660478+0000","last_clean":"2024-04-17T06:09:25.660478+0000","last_became_active":"2024-04-17T06:09:20.629335+0000","last_became_peered":"2024-04-17T06:09:20.629335+0000","last_unstale":"2024-04-17T06:09:25.660478+0000","last_undegraded":"2024-04-17T06:09:25.660478+0000","last_fullsized":"2024-04-17T06:09:25.660478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010974+0000","last_change":"2024-04-17T06:09:24.681484+0000","last_active":"2024-04-17T06:09:26.010974+0000","last_peered":"2024-04-17T06:09:26.010974+0000","last_clean":"2024-04-17T06:09:26.010974+0000","last_became_active":"2024-04-17T06:09:24.681074+0000","last_became_peered":"2024-04-17T06:09:24.681074+0000","last_unstale":"2024-04-17T06:09:26.010974+0000","last_undegraded":"2024-04-17T06:09:26.010974+0000","last_fullsized":"2024-04-17T06:09:26.010974+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:21.508074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011016+0000","last_change":"2024-04-17T06:09:19.629666+0000","last_active":"2024-04-17T06:09:26.011016+0000","last_peered":"2024-04-17T06:09:26.011016+0000","last_clean":"2024-04-17T06:09:26.011016+0000","last_became_active":"2024-04-17T06:09:19.629414+0000","last_became_peered":"2024-04-17T06:09:19.629414+0000","last_unstale":"2024-04-17T06:09:26.011016+0000","last_undegraded":"2024-04-17T06:09:26.011016+0000","last_fullsized":"2024-04-17T06:09:26.011016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660727+0000","last_change":"2024-04-17T06:09:20.629643+0000","last_active":"2024-04-17T06:09:25.660727+0000","last_peered":"2024-04-17T06:09:25.660727+0000","last_clean":"2024-04-17T06:09:25.660727+0000","last_became_active":"2024-04-17T06:09:20.629349+0000","last_became_peered":"2024-04-17T06:09:20.629349+0000","last_unstale":"2024-04-17T06:09:25.660727+0000","last_undegraded":"2024-04-17T06:09:25.660727+0000","last_fullsized":"2024-04-17T06:09:25.660727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:55.368933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011234+0000","last_change":"2024-04-17T06:09:24.673766+0000","last_active":"2024-04-17T06:09:26.011234+0000","last_peered":"2024-04-17T06:09:26.011234+0000","last_clean":"2024-04-17T06:09:26.011234+0000","last_became_active":"2024-04-17T06:09:24.673384+0000","last_became_peered":"2024-04-17T06:09:24.673384+0000","last_unstale":"2024-04-17T06:09:26.011234+0000","last_undegraded":"2024-04-17T06:09:26.011234+0000","last_fullsized":"2024-04-17T06:09:26.011234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:40.236016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659389+0000","last_change":"2024-04-17T06:09:20.643805+0000","last_active":"2024-04-17T06:09:25.659389+0000","last_peered":"2024-04-17T06:09:25.659389+0000","last_clean":"2024-04-17T06:09:25.659389+0000","last_became_active":"2024-04-17T06:09:20.643199+0000","last_became_peered":"2024-04-17T06:09:20.643199+0000","last_unstale":"2024-04-17T06:09:25.659389+0000","last_undegraded":"2024-04-17T06:09:25.659389+0000","last_fullsized":"2024-04-17T06:09:25.659389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:07.365672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011214+0000","last_change":"2024-04-17T06:09:19.626503+0000","last_active":"2024-04-17T06:09:26.011214+0000","last_peered":"2024-04-17T06:09:26.011214+0000","last_clean":"2024-04-17T06:09:26.011214+0000","last_became_active":"2024-04-17T06:09:19.625556+0000","last_became_peered":"2024-04-17T06:09:19.625556+0000","last_unstale":"2024-04-17T06:09:26.011214+0000","last_undegraded":"2024-04-17T06:09:26.011214+0000","last_fullsized":"2024-04-17T06:09:26.011214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010558+0000","last_change":"2024-04-17T06:09:24.680158+0000","last_active":"2024-04-17T06:09:26.010558+0000","last_peered":"2024-04-17T06:09:26.010558+0000","last_clean":"2024-04-17T06:09:26.010558+0000","last_became_active":"2024-04-17T06:09:24.679694+0000","last_became_peered":"2024-04-17T06:09:24.679694+0000","last_unstale":"2024-04-17T06:09:26.010558+0000","last_undegraded":"2024-04-17T06:09:26.010558+0000","last_fullsized":"2024-04-17T06:09:26.010558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.282988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011354+0000","last_change":"2024-04-17T06:09:20.638610+0000","last_active":"2024-04-17T06:09:26.011354+0000","last_peered":"2024-04-17T06:09:26.011354+0000","last_clean":"2024-04-17T06:09:26.011354+0000","last_became_active":"2024-04-17T06:09:20.638275+0000","last_became_peered":"2024-04-17T06:09:20.638275+0000","last_unstale":"2024-04-17T06:09:26.011354+0000","last_undegraded":"2024-04-17T06:09:26.011354+0000","last_fullsized":"2024-04-17T06:09:26.011354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011423+0000","last_change":"2024-04-17T06:09:19.626821+0000","last_active":"2024-04-17T06:09:26.011423+0000","last_peered":"2024-04-17T06:09:26.011423+0000","last_clean":"2024-04-17T06:09:26.011423+0000","last_became_active":"2024-04-17T06:09:19.626550+0000","last_became_peered":"2024-04-17T06:09:19.626550+0000","last_unstale":"2024-04-17T06:09:26.011423+0000","last_undegraded":"2024-04-17T06:09:26.011423+0000","last_fullsized":"2024-04-17T06:09:26.011423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010388+0000","last_change":"2024-04-17T06:09:24.680040+0000","last_active":"2024-04-17T06:09:26.010388+0000","last_peered":"2024-04-17T06:09:26.010388+0000","last_clean":"2024-04-17T06:09:26.010388+0000","last_became_active":"2024-04-17T06:09:24.679567+0000","last_became_peered":"2024-04-17T06:09:24.679567+0000","last_unstale":"2024-04-17T06:09:26.010388+0000","last_undegraded":"2024-04-17T06:09:26.010388+0000","last_fullsized":"2024-04-17T06:09:26.010388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:21.549944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010400+0000","last_change":"2024-04-17T06:09:20.645617+0000","last_active":"2024-04-17T06:09:26.010400+0000","last_peered":"2024-04-17T06:09:26.010400+0000","last_clean":"2024-04-17T06:09:26.010400+0000","last_became_active":"2024-04-17T06:09:20.645205+0000","last_became_peered":"2024-04-17T06:09:20.645205+0000","last_unstale":"2024-04-17T06:09:26.010400+0000","last_undegraded":"2024-04-17T06:09:26.010400+0000","last_fullsized":"2024-04-17T06:09:26.010400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010623+0000","last_change":"2024-04-17T06:09:19.623695+0000","last_active":"2024-04-17T06:09:26.010623+0000","last_peered":"2024-04-17T06:09:26.010623+0000","last_clean":"2024-04-17T06:09:26.010623+0000","last_became_active":"2024-04-17T06:09:19.623442+0000","last_became_peered":"2024-04-17T06:09:19.623442+0000","last_unstale":"2024-04-17T06:09:26.010623+0000","last_undegraded":"2024-04-17T06:09:26.010623+0000","last_fullsized":"2024-04-17T06:09:26.010623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:07.888042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010887+0000","last_change":"2024-04-17T06:09:24.990476+0000","last_active":"2024-04-17T06:09:26.010887+0000","last_peered":"2024-04-17T06:09:26.010887+0000","last_clean":"2024-04-17T06:09:26.010887+0000","last_became_active":"2024-04-17T06:09:24.990092+0000","last_became_peered":"2024-04-17T06:09:24.990092+0000","last_unstale":"2024-04-17T06:09:26.010887+0000","last_undegraded":"2024-04-17T06:09:26.010887+0000","last_fullsized":"2024-04-17T06:09:26.010887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010691+0000","last_change":"2024-04-17T06:09:20.633239+0000","last_active":"2024-04-17T06:09:26.010691+0000","last_peered":"2024-04-17T06:09:26.010691+0000","last_clean":"2024-04-17T06:09:26.010691+0000","last_became_active":"2024-04-17T06:09:20.632946+0000","last_became_peered":"2024-04-17T06:09:20.632946+0000","last_unstale":"2024-04-17T06:09:26.010691+0000","last_undegraded":"2024-04-17T06:09:26.010691+0000","last_fullsized":"2024-04-17T06:09:26.010691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:01.439932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010236+0000","last_change":"2024-04-17T06:09:19.632434+0000","last_active":"2024-04-17T06:09:26.010236+0000","last_peered":"2024-04-17T06:09:26.010236+0000","last_clean":"2024-04-17T06:09:26.010236+0000","last_became_active":"2024-04-17T06:09:19.632174+0000","last_became_peered":"2024-04-17T06:09:19.632174+0000","last_unstale":"2024-04-17T06:09:26.010236+0000","last_undegraded":"2024-04-17T06:09:26.010236+0000","last_fullsized":"2024-04-17T06:09:26.010236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:06.724008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010716+0000","last_change":"2024-04-17T06:09:24.680395+0000","last_active":"2024-04-17T06:09:26.010716+0000","last_peered":"2024-04-17T06:09:26.010716+0000","last_clean":"2024-04-17T06:09:26.010716+0000","last_became_active":"2024-04-17T06:09:24.679895+0000","last_became_peered":"2024-04-17T06:09:24.679895+0000","last_unstale":"2024-04-17T06:09:26.010716+0000","last_undegraded":"2024-04-17T06:09:26.010716+0000","last_fullsized":"2024-04-17T06:09:26.010716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:14.465883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012593+0000","last_change":"2024-04-17T06:09:20.638642+0000","last_active":"2024-04-17T06:09:26.012593+0000","last_peered":"2024-04-17T06:09:26.012593+0000","last_clean":"2024-04-17T06:09:26.012593+0000","last_became_active":"2024-04-17T06:09:20.638351+0000","last_became_peered":"2024-04-17T06:09:20.638351+0000","last_unstale":"2024-04-17T06:09:26.012593+0000","last_undegraded":"2024-04-17T06:09:26.012593+0000","last_fullsized":"2024-04-17T06:09:26.012593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660496+0000","last_change":"2024-04-17T06:09:19.622032+0000","last_active":"2024-04-17T06:09:25.660496+0000","last_peered":"2024-04-17T06:09:25.660496+0000","last_clean":"2024-04-17T06:09:25.660496+0000","last_became_active":"2024-04-17T06:09:19.621765+0000","last_became_peered":"2024-04-17T06:09:19.621765+0000","last_unstale":"2024-04-17T06:09:25.660496+0000","last_undegraded":"2024-04-17T06:09:25.660496+0000","last_fullsized":"2024-04-17T06:09:25.660496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010414+0000","last_change":"2024-04-17T06:09:24.990792+0000","last_active":"2024-04-17T06:09:26.010414+0000","last_peered":"2024-04-17T06:09:26.010414+0000","last_clean":"2024-04-17T06:09:26.010414+0000","last_became_active":"2024-04-17T06:09:24.990371+0000","last_became_peered":"2024-04-17T06:09:24.990371+0000","last_unstale":"2024-04-17T06:09:26.010414+0000","last_undegraded":"2024-04-17T06:09:26.010414+0000","last_fullsized":"2024-04-17T06:09:26.010414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:26.189133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011005+0000","last_change":"2024-04-17T06:09:20.648546+0000","last_active":"2024-04-17T06:09:26.011005+0000","last_peered":"2024-04-17T06:09:26.011005+0000","last_clean":"2024-04-17T06:09:26.011005+0000","last_became_active":"2024-04-17T06:09:20.647853+0000","last_became_peered":"2024-04-17T06:09:20.647853+0000","last_unstale":"2024-04-17T06:09:26.011005+0000","last_undegraded":"2024-04-17T06:09:26.011005+0000","last_fullsized":"2024-04-17T06:09:26.011005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011025+0000","last_change":"2024-04-17T06:09:19.627150+0000","last_active":"2024-04-17T06:09:26.011025+0000","last_peered":"2024-04-17T06:09:26.011025+0000","last_clean":"2024-04-17T06:09:26.011025+0000","last_became_active":"2024-04-17T06:09:19.626875+0000","last_became_peered":"2024-04-17T06:09:19.626875+0000","last_unstale":"2024-04-17T06:09:26.011025+0000","last_undegraded":"2024-04-17T06:09:26.011025+0000","last_fullsized":"2024-04-17T06:09:26.011025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010576+0000","last_change":"2024-04-17T06:09:24.992298+0000","last_active":"2024-04-17T06:09:26.010576+0000","last_peered":"2024-04-17T06:09:26.010576+0000","last_clean":"2024-04-17T06:09:26.010576+0000","last_became_active":"2024-04-17T06:09:24.991871+0000","last_became_peered":"2024-04-17T06:09:24.991871+0000","last_unstale":"2024-04-17T06:09:26.010576+0000","last_undegraded":"2024-04-17T06:09:26.010576+0000","last_fullsized":"2024-04-17T06:09:26.010576+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:43.835421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011016+0000","last_change":"2024-04-17T06:09:20.639206+0000","last_active":"2024-04-17T06:09:26.011016+0000","last_peered":"2024-04-17T06:09:26.011016+0000","last_clean":"2024-04-17T06:09:26.011016+0000","last_became_active":"2024-04-17T06:09:20.638954+0000","last_became_peered":"2024-04-17T06:09:20.638954+0000","last_unstale":"2024-04-17T06:09:26.011016+0000","last_undegraded":"2024-04-17T06:09:26.011016+0000","last_fullsized":"2024-04-17T06:09:26.011016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011535+0000","last_change":"2024-04-17T06:09:19.632347+0000","last_active":"2024-04-17T06:09:26.011535+0000","last_peered":"2024-04-17T06:09:26.011535+0000","last_clean":"2024-04-17T06:09:26.011535+0000","last_became_active":"2024-04-17T06:09:19.632117+0000","last_became_peered":"2024-04-17T06:09:19.632117+0000","last_unstale":"2024-04-17T06:09:26.011535+0000","last_undegraded":"2024-04-17T06:09:26.011535+0000","last_fullsized":"2024-04-17T06:09:26.011535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:07.888042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011384+0000","last_change":"2024-04-17T06:09:24.989732+0000","last_active":"2024-04-17T06:09:26.011384+0000","last_peered":"2024-04-17T06:09:26.011384+0000","last_clean":"2024-04-17T06:09:26.011384+0000","last_became_active":"2024-04-17T06:09:24.989320+0000","last_became_peered":"2024-04-17T06:09:24.989320+0000","last_unstale":"2024-04-17T06:09:26.011384+0000","last_undegraded":"2024-04-17T06:09:26.011384+0000","last_fullsized":"2024-04-17T06:09:26.011384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660737+0000","last_change":"2024-04-17T06:09:20.652167+0000","last_active":"2024-04-17T06:09:25.660737+0000","last_peered":"2024-04-17T06:09:25.660737+0000","last_clean":"2024-04-17T06:09:25.660737+0000","last_became_active":"2024-04-17T06:09:20.651908+0000","last_became_peered":"2024-04-17T06:09:20.651908+0000","last_unstale":"2024-04-17T06:09:25.660737+0000","last_undegraded":"2024-04-17T06:09:25.660737+0000","last_fullsized":"2024-04-17T06:09:25.660737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:58.610924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011096+0000","last_change":"2024-04-17T06:09:19.631750+0000","last_active":"2024-04-17T06:09:26.011096+0000","last_peered":"2024-04-17T06:09:26.011096+0000","last_clean":"2024-04-17T06:09:26.011096+0000","last_became_active":"2024-04-17T06:09:19.631463+0000","last_became_peered":"2024-04-17T06:09:19.631463+0000","last_unstale":"2024-04-17T06:09:26.011096+0000","last_undegraded":"2024-04-17T06:09:26.011096+0000","last_fullsized":"2024-04-17T06:09:26.011096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:07.389883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011105+0000","last_change":"2024-04-17T06:09:24.680787+0000","last_active":"2024-04-17T06:09:26.011105+0000","last_peered":"2024-04-17T06:09:26.011105+0000","last_clean":"2024-04-17T06:09:26.011105+0000","last_became_active":"2024-04-17T06:09:24.680279+0000","last_became_peered":"2024-04-17T06:09:24.680279+0000","last_unstale":"2024-04-17T06:09:26.011105+0000","last_undegraded":"2024-04-17T06:09:26.011105+0000","last_fullsized":"2024-04-17T06:09:26.011105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:25.702449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011262+0000","last_change":"2024-04-17T06:09:20.632789+0000","last_active":"2024-04-17T06:09:26.011262+0000","last_peered":"2024-04-17T06:09:26.011262+0000","last_clean":"2024-04-17T06:09:26.011262+0000","last_became_active":"2024-04-17T06:09:20.632199+0000","last_became_peered":"2024-04-17T06:09:20.632199+0000","last_unstale":"2024-04-17T06:09:26.011262+0000","last_undegraded":"2024-04-17T06:09:26.011262+0000","last_fullsized":"2024-04-17T06:09:26.011262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:08.393904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660530+0000","last_change":"2024-04-17T06:09:19.624615+0000","last_active":"2024-04-17T06:09:25.660530+0000","last_peered":"2024-04-17T06:09:25.660530+0000","last_clean":"2024-04-17T06:09:25.660530+0000","last_became_active":"2024-04-17T06:09:19.624333+0000","last_became_peered":"2024-04-17T06:09:19.624333+0000","last_unstale":"2024-04-17T06:09:25.660530+0000","last_undegraded":"2024-04-17T06:09:25.660530+0000","last_fullsized":"2024-04-17T06:09:25.660530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011059+0000","last_change":"2024-04-17T06:09:24.675868+0000","last_active":"2024-04-17T06:09:26.011059+0000","last_peered":"2024-04-17T06:09:26.011059+0000","last_clean":"2024-04-17T06:09:26.011059+0000","last_became_active":"2024-04-17T06:09:24.675269+0000","last_became_peered":"2024-04-17T06:09:24.675269+0000","last_unstale":"2024-04-17T06:09:26.011059+0000","last_undegraded":"2024-04-17T06:09:26.011059+0000","last_fullsized":"2024-04-17T06:09:26.011059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:02.650317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010687+0000","last_change":"2024-04-17T06:09:20.638626+0000","last_active":"2024-04-17T06:09:26.010687+0000","last_peered":"2024-04-17T06:09:26.010687+0000","last_clean":"2024-04-17T06:09:26.010687+0000","last_became_active":"2024-04-17T06:09:20.638339+0000","last_became_peered":"2024-04-17T06:09:20.638339+0000","last_unstale":"2024-04-17T06:09:26.010687+0000","last_undegraded":"2024-04-17T06:09:26.010687+0000","last_fullsized":"2024-04-17T06:09:26.010687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661071+0000","last_change":"2024-04-17T06:09:19.621292+0000","last_active":"2024-04-17T06:09:25.661071+0000","last_peered":"2024-04-17T06:09:25.661071+0000","last_clean":"2024-04-17T06:09:25.661071+0000","last_became_active":"2024-04-17T06:09:19.620827+0000","last_became_peered":"2024-04-17T06:09:19.620827+0000","last_unstale":"2024-04-17T06:09:25.661071+0000","last_undegraded":"2024-04-17T06:09:25.661071+0000","last_fullsized":"2024-04-17T06:09:25.661071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011064+0000","last_change":"2024-04-17T06:09:24.682574+0000","last_active":"2024-04-17T06:09:26.011064+0000","last_peered":"2024-04-17T06:09:26.011064+0000","last_clean":"2024-04-17T06:09:26.011064+0000","last_became_active":"2024-04-17T06:09:24.680550+0000","last_became_peered":"2024-04-17T06:09:24.680550+0000","last_unstale":"2024-04-17T06:09:26.011064+0000","last_undegraded":"2024-04-17T06:09:26.011064+0000","last_fullsized":"2024-04-17T06:09:26.011064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:20.183387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010301+0000","last_change":"2024-04-17T06:09:20.630308+0000","last_active":"2024-04-17T06:09:26.010301+0000","last_peered":"2024-04-17T06:09:26.010301+0000","last_clean":"2024-04-17T06:09:26.010301+0000","last_became_active":"2024-04-17T06:09:20.630052+0000","last_became_peered":"2024-04-17T06:09:20.630052+0000","last_unstale":"2024-04-17T06:09:26.010301+0000","last_undegraded":"2024-04-17T06:09:26.010301+0000","last_fullsized":"2024-04-17T06:09:26.010301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:53.394999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010349+0000","last_change":"2024-04-17T06:09:19.624643+0000","last_active":"2024-04-17T06:09:26.010349+0000","last_peered":"2024-04-17T06:09:26.010349+0000","last_clean":"2024-04-17T06:09:26.010349+0000","last_became_active":"2024-04-17T06:09:19.624270+0000","last_became_peered":"2024-04-17T06:09:19.624270+0000","last_unstale":"2024-04-17T06:09:26.010349+0000","last_undegraded":"2024-04-17T06:09:26.010349+0000","last_fullsized":"2024-04-17T06:09:26.010349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012635+0000","last_change":"2024-04-17T06:09:24.989917+0000","last_active":"2024-04-17T06:09:26.012635+0000","last_peered":"2024-04-17T06:09:26.012635+0000","last_clean":"2024-04-17T06:09:26.012635+0000","last_became_active":"2024-04-17T06:09:24.989308+0000","last_became_peered":"2024-04-17T06:09:24.989308+0000","last_unstale":"2024-04-17T06:09:26.012635+0000","last_undegraded":"2024-04-17T06:09:26.012635+0000","last_fullsized":"2024-04-17T06:09:26.012635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:14:19.804372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010303+0000","last_change":"2024-04-17T06:09:20.656161+0000","last_active":"2024-04-17T06:09:26.010303+0000","last_peered":"2024-04-17T06:09:26.010303+0000","last_clean":"2024-04-17T06:09:26.010303+0000","last_became_active":"2024-04-17T06:09:20.655897+0000","last_became_peered":"2024-04-17T06:09:20.655897+0000","last_unstale":"2024-04-17T06:09:26.010303+0000","last_undegraded":"2024-04-17T06:09:26.010303+0000","last_fullsized":"2024-04-17T06:09:26.010303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660641+0000","last_change":"2024-04-17T06:09:19.621223+0000","last_active":"2024-04-17T06:09:25.660641+0000","last_peered":"2024-04-17T06:09:25.660641+0000","last_clean":"2024-04-17T06:09:25.660641+0000","last_became_active":"2024-04-17T06:09:19.620409+0000","last_became_peered":"2024-04-17T06:09:19.620409+0000","last_unstale":"2024-04-17T06:09:25.660641+0000","last_undegraded":"2024-04-17T06:09:25.660641+0000","last_fullsized":"2024-04-17T06:09:25.660641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010841+0000","last_change":"2024-04-17T06:09:09.067035+0000","last_active":"2024-04-17T06:09:26.010841+0000","last_peered":"2024-04-17T06:09:26.010841+0000","last_clean":"2024-04-17T06:09:26.010841+0000","last_became_active":"2024-04-17T06:09:09.066130+0000","last_became_peered":"2024-04-17T06:09:09.066130+0000","last_unstale":"2024-04-17T06:09:26.010841+0000","last_undegraded":"2024-04-17T06:09:26.010841+0000","last_fullsized":"2024-04-17T06:09:26.010841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_clean_scrub_stamp":"2024-04-17T06:09:07.958875+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:03.277836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659861+0000","last_change":"2024-04-17T06:09:24.992008+0000","last_active":"2024-04-17T06:09:25.659861+0000","last_peered":"2024-04-17T06:09:25.659861+0000","last_clean":"2024-04-17T06:09:25.659861+0000","last_became_active":"2024-04-17T06:09:24.991448+0000","last_became_peered":"2024-04-17T06:09:24.991448+0000","last_unstale":"2024-04-17T06:09:25.659861+0000","last_undegraded":"2024-04-17T06:09:25.659861+0000","last_fullsized":"2024-04-17T06:09:25.659861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:15.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010640+0000","last_change":"2024-04-17T06:09:20.646916+0000","last_active":"2024-04-17T06:09:26.010640+0000","last_peered":"2024-04-17T06:09:26.010640+0000","last_clean":"2024-04-17T06:09:26.010640+0000","last_became_active":"2024-04-17T06:09:20.646069+0000","last_became_peered":"2024-04-17T06:09:20.646069+0000","last_unstale":"2024-04-17T06:09:26.010640+0000","last_undegraded":"2024-04-17T06:09:26.010640+0000","last_fullsized":"2024-04-17T06:09:26.010640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:11.442995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661446+0000","last_change":"2024-04-17T06:09:19.626825+0000","last_active":"2024-04-17T06:09:25.661446+0000","last_peered":"2024-04-17T06:09:25.661446+0000","last_clean":"2024-04-17T06:09:25.661446+0000","last_became_active":"2024-04-17T06:09:19.626549+0000","last_became_peered":"2024-04-17T06:09:19.626549+0000","last_unstale":"2024-04-17T06:09:25.661446+0000","last_undegraded":"2024-04-17T06:09:25.661446+0000","last_fullsized":"2024-04-17T06:09:25.661446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659329+0000","last_change":"2024-04-17T06:09:24.990665+0000","last_active":"2024-04-17T06:09:25.659329+0000","last_peered":"2024-04-17T06:09:25.659329+0000","last_clean":"2024-04-17T06:09:25.659329+0000","last_became_active":"2024-04-17T06:09:24.990262+0000","last_became_peered":"2024-04-17T06:09:24.990262+0000","last_unstale":"2024-04-17T06:09:25.659329+0000","last_undegraded":"2024-04-17T06:09:25.659329+0000","last_fullsized":"2024-04-17T06:09:25.659329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:21.508074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660652+0000","last_change":"2024-04-17T06:09:20.650264+0000","last_active":"2024-04-17T06:09:25.660652+0000","last_peered":"2024-04-17T06:09:25.660652+0000","last_clean":"2024-04-17T06:09:25.660652+0000","last_became_active":"2024-04-17T06:09:20.649792+0000","last_became_peered":"2024-04-17T06:09:20.649792+0000","last_unstale":"2024-04-17T06:09:25.660652+0000","last_undegraded":"2024-04-17T06:09:25.660652+0000","last_fullsized":"2024-04-17T06:09:25.660652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:36.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011681+0000","last_change":"2024-04-17T06:09:19.625656+0000","last_active":"2024-04-17T06:09:26.011681+0000","last_peered":"2024-04-17T06:09:26.011681+0000","last_clean":"2024-04-17T06:09:26.011681+0000","last_became_active":"2024-04-17T06:09:19.625408+0000","last_became_peered":"2024-04-17T06:09:19.625408+0000","last_unstale":"2024-04-17T06:09:26.011681+0000","last_undegraded":"2024-04-17T06:09:26.011681+0000","last_fullsized":"2024-04-17T06:09:26.011681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011611+0000","last_change":"2024-04-17T06:09:24.684050+0000","last_active":"2024-04-17T06:09:26.011611+0000","last_peered":"2024-04-17T06:09:26.011611+0000","last_clean":"2024-04-17T06:09:26.011611+0000","last_became_active":"2024-04-17T06:09:24.683688+0000","last_became_peered":"2024-04-17T06:09:24.683688+0000","last_unstale":"2024-04-17T06:09:26.011611+0000","last_undegraded":"2024-04-17T06:09:26.011611+0000","last_fullsized":"2024-04-17T06:09:26.011611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:02.650317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011613+0000","last_change":"2024-04-17T06:09:20.645936+0000","last_active":"2024-04-17T06:09:26.011613+0000","last_peered":"2024-04-17T06:09:26.011613+0000","last_clean":"2024-04-17T06:09:26.011613+0000","last_became_active":"2024-04-17T06:09:20.645453+0000","last_became_peered":"2024-04-17T06:09:20.645453+0000","last_unstale":"2024-04-17T06:09:26.011613+0000","last_undegraded":"2024-04-17T06:09:26.011613+0000","last_fullsized":"2024-04-17T06:09:26.011613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:21.663057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661396+0000","last_change":"2024-04-17T06:09:19.621217+0000","last_active":"2024-04-17T06:09:25.661396+0000","last_peered":"2024-04-17T06:09:25.661396+0000","last_clean":"2024-04-17T06:09:25.661396+0000","last_became_active":"2024-04-17T06:09:19.620846+0000","last_became_peered":"2024-04-17T06:09:19.620846+0000","last_unstale":"2024-04-17T06:09:25.661396+0000","last_undegraded":"2024-04-17T06:09:25.661396+0000","last_fullsized":"2024-04-17T06:09:25.661396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659685+0000","last_change":"2024-04-17T06:09:24.991134+0000","last_active":"2024-04-17T06:09:25.659685+0000","last_peered":"2024-04-17T06:09:25.659685+0000","last_clean":"2024-04-17T06:09:25.659685+0000","last_became_active":"2024-04-17T06:09:24.990742+0000","last_became_peered":"2024-04-17T06:09:24.990742+0000","last_unstale":"2024-04-17T06:09:25.659685+0000","last_undegraded":"2024-04-17T06:09:25.659685+0000","last_fullsized":"2024-04-17T06:09:25.659685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:54.153325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011276+0000","last_change":"2024-04-17T06:09:20.645660+0000","last_active":"2024-04-17T06:09:26.011276+0000","last_peered":"2024-04-17T06:09:26.011276+0000","last_clean":"2024-04-17T06:09:26.011276+0000","last_became_active":"2024-04-17T06:09:20.645370+0000","last_became_peered":"2024-04-17T06:09:20.645370+0000","last_unstale":"2024-04-17T06:09:26.011276+0000","last_undegraded":"2024-04-17T06:09:26.011276+0000","last_fullsized":"2024-04-17T06:09:26.011276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:36.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011721+0000","last_change":"2024-04-17T06:09:19.633064+0000","last_active":"2024-04-17T06:09:26.011721+0000","last_peered":"2024-04-17T06:09:26.011721+0000","last_clean":"2024-04-17T06:09:26.011721+0000","last_became_active":"2024-04-17T06:09:19.632759+0000","last_became_peered":"2024-04-17T06:09:19.632759+0000","last_unstale":"2024-04-17T06:09:26.011721+0000","last_undegraded":"2024-04-17T06:09:26.011721+0000","last_fullsized":"2024-04-17T06:09:26.011721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:58.547080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010213+0000","last_change":"2024-04-17T06:09:24.676408+0000","last_active":"2024-04-17T06:09:26.010213+0000","last_peered":"2024-04-17T06:09:26.010213+0000","last_clean":"2024-04-17T06:09:26.010213+0000","last_became_active":"2024-04-17T06:09:24.675996+0000","last_became_peered":"2024-04-17T06:09:24.675996+0000","last_unstale":"2024-04-17T06:09:26.010213+0000","last_undegraded":"2024-04-17T06:09:26.010213+0000","last_fullsized":"2024-04-17T06:09:26.010213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:14.169938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010896+0000","last_change":"2024-04-17T06:09:20.645388+0000","last_active":"2024-04-17T06:09:26.010896+0000","last_peered":"2024-04-17T06:09:26.010896+0000","last_clean":"2024-04-17T06:09:26.010896+0000","last_became_active":"2024-04-17T06:09:20.645091+0000","last_became_peered":"2024-04-17T06:09:20.645091+0000","last_unstale":"2024-04-17T06:09:26.010896+0000","last_undegraded":"2024-04-17T06:09:26.010896+0000","last_fullsized":"2024-04-17T06:09:26.010896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660949+0000","last_change":"2024-04-17T06:09:19.624671+0000","last_active":"2024-04-17T06:09:25.660949+0000","last_peered":"2024-04-17T06:09:25.660949+0000","last_clean":"2024-04-17T06:09:25.660949+0000","last_became_active":"2024-04-17T06:09:19.624360+0000","last_became_peered":"2024-04-17T06:09:19.624360+0000","last_unstale":"2024-04-17T06:09:25.660949+0000","last_undegraded":"2024-04-17T06:09:25.660949+0000","last_fullsized":"2024-04-17T06:09:25.660949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:35.753743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012475+0000","last_change":"2024-04-17T06:09:24.677224+0000","last_active":"2024-04-17T06:09:26.012475+0000","last_peered":"2024-04-17T06:09:26.012475+0000","last_clean":"2024-04-17T06:09:26.012475+0000","last_became_active":"2024-04-17T06:09:24.676834+0000","last_became_peered":"2024-04-17T06:09:24.676834+0000","last_unstale":"2024-04-17T06:09:26.012475+0000","last_undegraded":"2024-04-17T06:09:26.012475+0000","last_fullsized":"2024-04-17T06:09:26.012475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:25.702449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659412+0000","last_change":"2024-04-17T06:09:20.640720+0000","last_active":"2024-04-17T06:09:25.659412+0000","last_peered":"2024-04-17T06:09:25.659412+0000","last_clean":"2024-04-17T06:09:25.659412+0000","last_became_active":"2024-04-17T06:09:20.639180+0000","last_became_peered":"2024-04-17T06:09:20.639180+0000","last_unstale":"2024-04-17T06:09:25.659412+0000","last_undegraded":"2024-04-17T06:09:25.659412+0000","last_fullsized":"2024-04-17T06:09:25.659412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659510+0000","last_change":"2024-04-17T06:09:19.623937+0000","last_active":"2024-04-17T06:09:25.659510+0000","last_peered":"2024-04-17T06:09:25.659510+0000","last_clean":"2024-04-17T06:09:25.659510+0000","last_became_active":"2024-04-17T06:09:19.623613+0000","last_became_peered":"2024-04-17T06:09:19.623613+0000","last_unstale":"2024-04-17T06:09:25.659510+0000","last_undegraded":"2024-04-17T06:09:25.659510+0000","last_fullsized":"2024-04-17T06:09:25.659510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010862+0000","last_change":"2024-04-17T06:09:24.991659+0000","last_active":"2024-04-17T06:09:26.010862+0000","last_peered":"2024-04-17T06:09:26.010862+0000","last_clean":"2024-04-17T06:09:26.010862+0000","last_became_active":"2024-04-17T06:09:24.991305+0000","last_became_peered":"2024-04-17T06:09:24.991305+0000","last_unstale":"2024-04-17T06:09:26.010862+0000","last_undegraded":"2024-04-17T06:09:26.010862+0000","last_fullsized":"2024-04-17T06:09:26.010862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:22.763944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010997+0000","last_change":"2024-04-17T06:09:20.647491+0000","last_active":"2024-04-17T06:09:26.010997+0000","last_peered":"2024-04-17T06:09:26.010997+0000","last_clean":"2024-04-17T06:09:26.010997+0000","last_became_active":"2024-04-17T06:09:20.647222+0000","last_became_peered":"2024-04-17T06:09:20.647222+0000","last_unstale":"2024-04-17T06:09:26.010997+0000","last_undegraded":"2024-04-17T06:09:26.010997+0000","last_fullsized":"2024-04-17T06:09:26.010997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660425+0000","last_change":"2024-04-17T06:09:19.622734+0000","last_active":"2024-04-17T06:09:25.660425+0000","last_peered":"2024-04-17T06:09:25.660425+0000","last_clean":"2024-04-17T06:09:25.660425+0000","last_became_active":"2024-04-17T06:09:19.622401+0000","last_became_peered":"2024-04-17T06:09:19.622401+0000","last_unstale":"2024-04-17T06:09:25.660425+0000","last_undegraded":"2024-04-17T06:09:25.660425+0000","last_fullsized":"2024-04-17T06:09:25.660425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012343+0000","last_change":"2024-04-17T06:09:24.989713+0000","last_active":"2024-04-17T06:09:26.012343+0000","last_peered":"2024-04-17T06:09:26.012343+0000","last_clean":"2024-04-17T06:09:26.012343+0000","last_became_active":"2024-04-17T06:09:24.989311+0000","last_became_peered":"2024-04-17T06:09:24.989311+0000","last_unstale":"2024-04-17T06:09:26.012343+0000","last_undegraded":"2024-04-17T06:09:26.012343+0000","last_fullsized":"2024-04-17T06:09:26.012343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:09.035921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010838+0000","last_change":"2024-04-17T06:09:20.648156+0000","last_active":"2024-04-17T06:09:26.010838+0000","last_peered":"2024-04-17T06:09:26.010838+0000","last_clean":"2024-04-17T06:09:26.010838+0000","last_became_active":"2024-04-17T06:09:20.647878+0000","last_became_peered":"2024-04-17T06:09:20.647878+0000","last_unstale":"2024-04-17T06:09:26.010838+0000","last_undegraded":"2024-04-17T06:09:26.010838+0000","last_fullsized":"2024-04-17T06:09:26.010838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011779+0000","last_change":"2024-04-17T06:09:19.631943+0000","last_active":"2024-04-17T06:09:26.011779+0000","last_peered":"2024-04-17T06:09:26.011779+0000","last_clean":"2024-04-17T06:09:26.011779+0000","last_became_active":"2024-04-17T06:09:19.631287+0000","last_became_peered":"2024-04-17T06:09:19.631287+0000","last_unstale":"2024-04-17T06:09:26.011779+0000","last_undegraded":"2024-04-17T06:09:26.011779+0000","last_fullsized":"2024-04-17T06:09:26.011779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:44.893832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660591+0000","last_change":"2024-04-17T06:09:24.989665+0000","last_active":"2024-04-17T06:09:25.660591+0000","last_peered":"2024-04-17T06:09:25.660591+0000","last_clean":"2024-04-17T06:09:25.660591+0000","last_became_active":"2024-04-17T06:09:24.989241+0000","last_became_peered":"2024-04-17T06:09:24.989241+0000","last_unstale":"2024-04-17T06:09:25.660591+0000","last_undegraded":"2024-04-17T06:09:25.660591+0000","last_fullsized":"2024-04-17T06:09:25.660591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:43.835421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010532+0000","last_change":"2024-04-17T06:09:20.647147+0000","last_active":"2024-04-17T06:09:26.010532+0000","last_peered":"2024-04-17T06:09:26.010532+0000","last_clean":"2024-04-17T06:09:26.010532+0000","last_became_active":"2024-04-17T06:09:20.646657+0000","last_became_peered":"2024-04-17T06:09:20.646657+0000","last_unstale":"2024-04-17T06:09:26.010532+0000","last_undegraded":"2024-04-17T06:09:26.010532+0000","last_fullsized":"2024-04-17T06:09:26.010532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:30.115357+0000","last_change":"2024-04-17T06:09:19.629001+0000","last_active":"2024-04-17T06:09:30.115357+0000","last_peered":"2024-04-17T06:09:30.115357+0000","last_clean":"2024-04-17T06:09:30.115357+0000","last_became_active":"2024-04-17T06:09:19.628654+0000","last_became_peered":"2024-04-17T06:09:19.628654+0000","last_unstale":"2024-04-17T06:09:30.115357+0000","last_undegraded":"2024-04-17T06:09:30.115357+0000","last_fullsized":"2024-04-17T06:09:30.115357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011319+0000","last_change":"2024-04-17T06:09:24.682555+0000","last_active":"2024-04-17T06:09:26.011319+0000","last_peered":"2024-04-17T06:09:26.011319+0000","last_clean":"2024-04-17T06:09:26.011319+0000","last_became_active":"2024-04-17T06:09:24.680576+0000","last_became_peered":"2024-04-17T06:09:24.680576+0000","last_unstale":"2024-04-17T06:09:26.011319+0000","last_undegraded":"2024-04-17T06:09:26.011319+0000","last_fullsized":"2024-04-17T06:09:26.011319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:14:19.804372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010916+0000","last_change":"2024-04-17T06:09:20.644222+0000","last_active":"2024-04-17T06:09:26.010916+0000","last_peered":"2024-04-17T06:09:26.010916+0000","last_clean":"2024-04-17T06:09:26.010916+0000","last_became_active":"2024-04-17T06:09:20.643862+0000","last_became_peered":"2024-04-17T06:09:20.643862+0000","last_unstale":"2024-04-17T06:09:26.010916+0000","last_undegraded":"2024-04-17T06:09:26.010916+0000","last_fullsized":"2024-04-17T06:09:26.010916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:07.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011583+0000","last_change":"2024-04-17T06:09:19.628512+0000","last_active":"2024-04-17T06:09:26.011583+0000","last_peered":"2024-04-17T06:09:26.011583+0000","last_clean":"2024-04-17T06:09:26.011583+0000","last_became_active":"2024-04-17T06:09:19.628227+0000","last_became_peered":"2024-04-17T06:09:19.628227+0000","last_unstale":"2024-04-17T06:09:26.011583+0000","last_undegraded":"2024-04-17T06:09:26.011583+0000","last_fullsized":"2024-04-17T06:09:26.011583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011193+0000","last_change":"2024-04-17T06:09:24.682017+0000","last_active":"2024-04-17T06:09:26.011193+0000","last_peered":"2024-04-17T06:09:26.011193+0000","last_clean":"2024-04-17T06:09:26.011193+0000","last_became_active":"2024-04-17T06:09:24.681637+0000","last_became_peered":"2024-04-17T06:09:24.681637+0000","last_unstale":"2024-04-17T06:09:26.011193+0000","last_undegraded":"2024-04-17T06:09:26.011193+0000","last_fullsized":"2024-04-17T06:09:26.011193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:15.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012238+0000","last_change":"2024-04-17T06:09:20.638985+0000","last_active":"2024-04-17T06:09:26.012238+0000","last_peered":"2024-04-17T06:09:26.012238+0000","last_clean":"2024-04-17T06:09:26.012238+0000","last_became_active":"2024-04-17T06:09:20.638132+0000","last_became_peered":"2024-04-17T06:09:20.638132+0000","last_unstale":"2024-04-17T06:09:26.012238+0000","last_undegraded":"2024-04-17T06:09:26.012238+0000","last_fullsized":"2024-04-17T06:09:26.012238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660523+0000","last_change":"2024-04-17T06:09:19.621789+0000","last_active":"2024-04-17T06:09:25.660523+0000","last_peered":"2024-04-17T06:09:25.660523+0000","last_clean":"2024-04-17T06:09:25.660523+0000","last_became_active":"2024-04-17T06:09:19.621506+0000","last_became_peered":"2024-04-17T06:09:19.621506+0000","last_unstale":"2024-04-17T06:09:25.660523+0000","last_undegraded":"2024-04-17T06:09:25.660523+0000","last_fullsized":"2024-04-17T06:09:25.660523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010264+0000","last_change":"2024-04-17T06:09:24.676641+0000","last_active":"2024-04-17T06:09:26.010264+0000","last_peered":"2024-04-17T06:09:26.010264+0000","last_clean":"2024-04-17T06:09:26.010264+0000","last_became_active":"2024-04-17T06:09:24.676251+0000","last_became_peered":"2024-04-17T06:09:24.676251+0000","last_unstale":"2024-04-17T06:09:26.010264+0000","last_undegraded":"2024-04-17T06:09:26.010264+0000","last_fullsized":"2024-04-17T06:09:26.010264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:54.153325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660474+0000","last_change":"2024-04-17T06:09:20.643983+0000","last_active":"2024-04-17T06:09:25.660474+0000","last_peered":"2024-04-17T06:09:25.660474+0000","last_clean":"2024-04-17T06:09:25.660474+0000","last_became_active":"2024-04-17T06:09:20.643704+0000","last_became_peered":"2024-04-17T06:09:20.643704+0000","last_unstale":"2024-04-17T06:09:25.660474+0000","last_undegraded":"2024-04-17T06:09:25.660474+0000","last_fullsized":"2024-04-17T06:09:25.660474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660308+0000","last_change":"2024-04-17T06:09:19.626344+0000","last_active":"2024-04-17T06:09:25.660308+0000","last_peered":"2024-04-17T06:09:25.660308+0000","last_clean":"2024-04-17T06:09:25.660308+0000","last_became_active":"2024-04-17T06:09:19.626060+0000","last_became_peered":"2024-04-17T06:09:19.626060+0000","last_unstale":"2024-04-17T06:09:25.660308+0000","last_undegraded":"2024-04-17T06:09:25.660308+0000","last_fullsized":"2024-04-17T06:09:25.660308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011948+0000","last_change":"2024-04-17T06:09:24.990709+0000","last_active":"2024-04-17T06:09:26.011948+0000","last_peered":"2024-04-17T06:09:26.011948+0000","last_clean":"2024-04-17T06:09:26.011948+0000","last_became_active":"2024-04-17T06:09:24.990327+0000","last_became_peered":"2024-04-17T06:09:24.990327+0000","last_unstale":"2024-04-17T06:09:26.011948+0000","last_undegraded":"2024-04-17T06:09:26.011948+0000","last_fullsized":"2024-04-17T06:09:26.011948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:26:33.123735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012474+0000","last_change":"2024-04-17T06:09:24.994579+0000","last_active":"2024-04-17T06:09:26.012474+0000","last_peered":"2024-04-17T06:09:26.012474+0000","last_clean":"2024-04-17T06:09:26.012474+0000","last_became_active":"2024-04-17T06:09:24.994172+0000","last_became_peered":"2024-04-17T06:09:24.994172+0000","last_unstale":"2024-04-17T06:09:26.012474+0000","last_undegraded":"2024-04-17T06:09:26.012474+0000","last_fullsized":"2024-04-17T06:09:26.012474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:20.183387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010445+0000","last_change":"2024-04-17T06:09:20.646244+0000","last_active":"2024-04-17T06:09:26.010445+0000","last_peered":"2024-04-17T06:09:26.010445+0000","last_clean":"2024-04-17T06:09:26.010445+0000","last_became_active":"2024-04-17T06:09:20.645899+0000","last_became_peered":"2024-04-17T06:09:20.645899+0000","last_unstale":"2024-04-17T06:09:26.010445+0000","last_undegraded":"2024-04-17T06:09:26.010445+0000","last_fullsized":"2024-04-17T06:09:26.010445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659810+0000","last_change":"2024-04-17T06:09:19.621004+0000","last_active":"2024-04-17T06:09:25.659810+0000","last_peered":"2024-04-17T06:09:25.659810+0000","last_clean":"2024-04-17T06:09:25.659810+0000","last_became_active":"2024-04-17T06:09:19.620503+0000","last_became_peered":"2024-04-17T06:09:19.620503+0000","last_unstale":"2024-04-17T06:09:25.659810+0000","last_undegraded":"2024-04-17T06:09:25.659810+0000","last_fullsized":"2024-04-17T06:09:25.659810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660166+0000","last_change":"2024-04-17T06:09:24.674536+0000","last_active":"2024-04-17T06:09:25.660166+0000","last_peered":"2024-04-17T06:09:25.660166+0000","last_clean":"2024-04-17T06:09:25.660166+0000","last_became_active":"2024-04-17T06:09:24.674159+0000","last_became_peered":"2024-04-17T06:09:24.674159+0000","last_unstale":"2024-04-17T06:09:25.660166+0000","last_undegraded":"2024-04-17T06:09:25.660166+0000","last_fullsized":"2024-04-17T06:09:25.660166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:40.236016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010314+0000","last_change":"2024-04-17T06:09:20.638332+0000","last_active":"2024-04-17T06:09:26.010314+0000","last_peered":"2024-04-17T06:09:26.010314+0000","last_clean":"2024-04-17T06:09:26.010314+0000","last_became_active":"2024-04-17T06:09:20.638096+0000","last_became_peered":"2024-04-17T06:09:20.638096+0000","last_unstale":"2024-04-17T06:09:26.010314+0000","last_undegraded":"2024-04-17T06:09:26.010314+0000","last_fullsized":"2024-04-17T06:09:26.010314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012032+0000","last_change":"2024-04-17T06:09:19.627827+0000","last_active":"2024-04-17T06:09:26.012032+0000","last_peered":"2024-04-17T06:09:26.012032+0000","last_clean":"2024-04-17T06:09:26.012032+0000","last_became_active":"2024-04-17T06:09:19.627529+0000","last_became_peered":"2024-04-17T06:09:19.627529+0000","last_unstale":"2024-04-17T06:09:26.012032+0000","last_undegraded":"2024-04-17T06:09:26.012032+0000","last_fullsized":"2024-04-17T06:09:26.012032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659711+0000","last_change":"2024-04-17T06:09:19.623047+0000","last_active":"2024-04-17T06:09:25.659711+0000","last_peered":"2024-04-17T06:09:25.659711+0000","last_clean":"2024-04-17T06:09:25.659711+0000","last_became_active":"2024-04-17T06:09:19.622700+0000","last_became_peered":"2024-04-17T06:09:19.622700+0000","last_unstale":"2024-04-17T06:09:25.659711+0000","last_undegraded":"2024-04-17T06:09:25.659711+0000","last_fullsized":"2024-04-17T06:09:25.659711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011732+0000","last_change":"2024-04-17T06:09:20.630941+0000","last_active":"2024-04-17T06:09:26.011732+0000","last_peered":"2024-04-17T06:09:26.011732+0000","last_clean":"2024-04-17T06:09:26.011732+0000","last_became_active":"2024-04-17T06:09:20.630680+0000","last_became_peered":"2024-04-17T06:09:20.630680+0000","last_unstale":"2024-04-17T06:09:26.011732+0000","last_undegraded":"2024-04-17T06:09:26.011732+0000","last_fullsized":"2024-04-17T06:09:26.011732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661370+0000","last_change":"2024-04-17T06:09:24.668919+0000","last_active":"2024-04-17T06:09:25.661370+0000","last_peered":"2024-04-17T06:09:25.661370+0000","last_clean":"2024-04-17T06:09:25.661370+0000","last_became_active":"2024-04-17T06:09:24.668456+0000","last_became_peered":"2024-04-17T06:09:24.668456+0000","last_unstale":"2024-04-17T06:09:25.661370+0000","last_undegraded":"2024-04-17T06:09:25.661370+0000","last_fullsized":"2024-04-17T06:09:25.661370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:57.434392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011442+0000","last_change":"2024-04-17T06:09:19.622862+0000","last_active":"2024-04-17T06:09:26.011442+0000","last_peered":"2024-04-17T06:09:26.011442+0000","last_clean":"2024-04-17T06:09:26.011442+0000","last_became_active":"2024-04-17T06:09:19.622514+0000","last_became_peered":"2024-04-17T06:09:19.622514+0000","last_unstale":"2024-04-17T06:09:26.011442+0000","last_undegraded":"2024-04-17T06:09:26.011442+0000","last_fullsized":"2024-04-17T06:09:26.011442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011374+0000","last_change":"2024-04-17T06:09:20.638095+0000","last_active":"2024-04-17T06:09:26.011374+0000","last_peered":"2024-04-17T06:09:26.011374+0000","last_clean":"2024-04-17T06:09:26.011374+0000","last_became_active":"2024-04-17T06:09:20.637805+0000","last_became_peered":"2024-04-17T06:09:20.637805+0000","last_unstale":"2024-04-17T06:09:26.011374+0000","last_undegraded":"2024-04-17T06:09:26.011374+0000","last_fullsized":"2024-04-17T06:09:26.011374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:48.980595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012082+0000","last_change":"2024-04-17T06:09:24.675683+0000","last_active":"2024-04-17T06:09:26.012082+0000","last_peered":"2024-04-17T06:09:26.012082+0000","last_clean":"2024-04-17T06:09:26.012082+0000","last_became_active":"2024-04-17T06:09:24.675309+0000","last_became_peered":"2024-04-17T06:09:24.675309+0000","last_unstale":"2024-04-17T06:09:26.012082+0000","last_undegraded":"2024-04-17T06:09:26.012082+0000","last_fullsized":"2024-04-17T06:09:26.012082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:10.888244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659223+0000","last_change":"2024-04-17T06:09:20.640987+0000","last_active":"2024-04-17T06:09:25.659223+0000","last_peered":"2024-04-17T06:09:25.659223+0000","last_clean":"2024-04-17T06:09:25.659223+0000","last_became_active":"2024-04-17T06:09:20.640396+0000","last_became_peered":"2024-04-17T06:09:20.640396+0000","last_unstale":"2024-04-17T06:09:25.659223+0000","last_undegraded":"2024-04-17T06:09:25.659223+0000","last_fullsized":"2024-04-17T06:09:25.659223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660950+0000","last_change":"2024-04-17T06:09:20.642450+0000","last_active":"2024-04-17T06:09:25.660950+0000","last_peered":"2024-04-17T06:09:25.660950+0000","last_clean":"2024-04-17T06:09:25.660950+0000","last_became_active":"2024-04-17T06:09:20.639465+0000","last_became_peered":"2024-04-17T06:09:20.639465+0000","last_unstale":"2024-04-17T06:09:25.660950+0000","last_undegraded":"2024-04-17T06:09:25.660950+0000","last_fullsized":"2024-04-17T06:09:25.660950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:08.393904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660526+0000","last_change":"2024-04-17T06:09:20.652419+0000","last_active":"2024-04-17T06:09:25.660526+0000","last_peered":"2024-04-17T06:09:25.660526+0000","last_clean":"2024-04-17T06:09:25.660526+0000","last_became_active":"2024-04-17T06:09:20.652139+0000","last_became_peered":"2024-04-17T06:09:20.652139+0000","last_unstale":"2024-04-17T06:09:25.660526+0000","last_undegraded":"2024-04-17T06:09:25.660526+0000","last_fullsized":"2024-04-17T06:09:25.660526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011662+0000","last_change":"2024-04-17T06:09:20.636986+0000","last_active":"2024-04-17T06:09:26.011662+0000","last_peered":"2024-04-17T06:09:26.011662+0000","last_clean":"2024-04-17T06:09:26.011662+0000","last_became_active":"2024-04-17T06:09:20.636761+0000","last_became_peered":"2024-04-17T06:09:20.636761+0000","last_unstale":"2024-04-17T06:09:26.011662+0000","last_undegraded":"2024-04-17T06:09:26.011662+0000","last_fullsized":"2024-04-17T06:09:26.011662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012351+0000","last_change":"2024-04-17T06:09:20.646197+0000","last_active":"2024-04-17T06:09:26.012351+0000","last_peered":"2024-04-17T06:09:26.012351+0000","last_clean":"2024-04-17T06:09:26.012351+0000","last_became_active":"2024-04-17T06:09:20.645965+0000","last_became_peered":"2024-04-17T06:09:20.645965+0000","last_unstale":"2024-04-17T06:09:26.012351+0000","last_undegraded":"2024-04-17T06:09:26.012351+0000","last_fullsized":"2024-04-17T06:09:26.012351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011415+0000","last_change":"2024-04-17T06:09:20.644696+0000","last_active":"2024-04-17T06:09:26.011415+0000","last_peered":"2024-04-17T06:09:26.011415+0000","last_clean":"2024-04-17T06:09:26.011415+0000","last_became_active":"2024-04-17T06:09:20.644322+0000","last_became_peered":"2024-04-17T06:09:20.644322+0000","last_unstale":"2024-04-17T06:09:26.011415+0000","last_undegraded":"2024-04-17T06:09:26.011415+0000","last_fullsized":"2024-04-17T06:09:26.011415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660919+0000","last_change":"2024-04-17T06:09:20.650016+0000","last_active":"2024-04-17T06:09:25.660919+0000","last_peered":"2024-04-17T06:09:25.660919+0000","last_clean":"2024-04-17T06:09:25.660919+0000","last_became_active":"2024-04-17T06:09:20.649252+0000","last_became_peered":"2024-04-17T06:09:20.649252+0000","last_unstale":"2024-04-17T06:09:25.660919+0000","last_undegraded":"2024-04-17T06:09:25.660919+0000","last_fullsized":"2024-04-17T06:09:25.660919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010916+0000","last_change":"2024-04-17T06:09:20.646003+0000","last_active":"2024-04-17T06:09:26.010916+0000","last_peered":"2024-04-17T06:09:26.010916+0000","last_clean":"2024-04-17T06:09:26.010916+0000","last_became_active":"2024-04-17T06:09:20.645732+0000","last_became_peered":"2024-04-17T06:09:20.645732+0000","last_unstale":"2024-04-17T06:09:26.010916+0000","last_undegraded":"2024-04-17T06:09:26.010916+0000","last_fullsized":"2024-04-17T06:09:26.010916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659748+0000","last_change":"2024-04-17T06:09:20.650803+0000","last_active":"2024-04-17T06:09:25.659748+0000","last_peered":"2024-04-17T06:09:25.659748+0000","last_clean":"2024-04-17T06:09:25.659748+0000","last_became_active":"2024-04-17T06:09:20.649856+0000","last_became_peered":"2024-04-17T06:09:20.649856+0000","last_unstale":"2024-04-17T06:09:25.659748+0000","last_undegraded":"2024-04-17T06:09:25.659748+0000","last_fullsized":"2024-04-17T06:09:25.659748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011942+0000","last_change":"2024-04-17T06:09:20.643474+0000","last_active":"2024-04-17T06:09:26.011942+0000","last_peered":"2024-04-17T06:09:26.011942+0000","last_clean":"2024-04-17T06:09:26.011942+0000","last_became_active":"2024-04-17T06:09:20.643125+0000","last_became_peered":"2024-04-17T06:09:20.643125+0000","last_unstale":"2024-04-17T06:09:26.011942+0000","last_undegraded":"2024-04-17T06:09:26.011942+0000","last_fullsized":"2024-04-17T06:09:26.011942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660299+0000","last_change":"2024-04-17T06:09:20.646984+0000","last_active":"2024-04-17T06:09:25.660299+0000","last_peered":"2024-04-17T06:09:25.660299+0000","last_clean":"2024-04-17T06:09:25.660299+0000","last_became_active":"2024-04-17T06:09:20.646731+0000","last_became_peered":"2024-04-17T06:09:20.646731+0000","last_unstale":"2024-04-17T06:09:25.660299+0000","last_undegraded":"2024-04-17T06:09:25.660299+0000","last_fullsized":"2024-04-17T06:09:25.660299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:11.442995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011435+0000","last_change":"2024-04-17T06:09:20.648582+0000","last_active":"2024-04-17T06:09:26.011435+0000","last_peered":"2024-04-17T06:09:26.011435+0000","last_clean":"2024-04-17T06:09:26.011435+0000","last_became_active":"2024-04-17T06:09:20.648345+0000","last_became_peered":"2024-04-17T06:09:20.648345+0000","last_unstale":"2024-04-17T06:09:26.011435+0000","last_undegraded":"2024-04-17T06:09:26.011435+0000","last_fullsized":"2024-04-17T06:09:26.011435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660359+0000","last_change":"2024-04-17T06:09:20.647687+0000","last_active":"2024-04-17T06:09:25.660359+0000","last_peered":"2024-04-17T06:09:25.660359+0000","last_clean":"2024-04-17T06:09:25.660359+0000","last_became_active":"2024-04-17T06:09:20.646672+0000","last_became_peered":"2024-04-17T06:09:20.646672+0000","last_unstale":"2024-04-17T06:09:25.660359+0000","last_undegraded":"2024-04-17T06:09:25.660359+0000","last_fullsized":"2024-04-17T06:09:25.660359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011526+0000","last_change":"2024-04-17T06:09:20.646327+0000","last_active":"2024-04-17T06:09:26.011526+0000","last_peered":"2024-04-17T06:09:26.011526+0000","last_clean":"2024-04-17T06:09:26.011526+0000","last_became_active":"2024-04-17T06:09:20.645768+0000","last_became_peered":"2024-04-17T06:09:20.645768+0000","last_unstale":"2024-04-17T06:09:26.011526+0000","last_undegraded":"2024-04-17T06:09:26.011526+0000","last_fullsized":"2024-04-17T06:09:26.011526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:55.368933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661388+0000","last_change":"2024-04-17T06:09:20.638979+0000","last_active":"2024-04-17T06:09:25.661388+0000","last_peered":"2024-04-17T06:09:25.661388+0000","last_clean":"2024-04-17T06:09:25.661388+0000","last_became_active":"2024-04-17T06:09:20.638679+0000","last_became_peered":"2024-04-17T06:09:20.638679+0000","last_unstale":"2024-04-17T06:09:25.661388+0000","last_undegraded":"2024-04-17T06:09:25.661388+0000","last_fullsized":"2024-04-17T06:09:25.661388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:08.892063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661039+0000","last_change":"2024-04-17T06:09:20.647598+0000","last_active":"2024-04-17T06:09:25.661039+0000","last_peered":"2024-04-17T06:09:25.661039+0000","last_clean":"2024-04-17T06:09:25.661039+0000","last_became_active":"2024-04-17T06:09:20.647352+0000","last_became_peered":"2024-04-17T06:09:20.647352+0000","last_unstale":"2024-04-17T06:09:25.661039+0000","last_undegraded":"2024-04-17T06:09:25.661039+0000","last_fullsized":"2024-04-17T06:09:25.661039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:16.143995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659697+0000","last_change":"2024-04-17T06:09:20.634824+0000","last_active":"2024-04-17T06:09:25.659697+0000","last_peered":"2024-04-17T06:09:25.659697+0000","last_clean":"2024-04-17T06:09:25.659697+0000","last_became_active":"2024-04-17T06:09:20.634526+0000","last_became_peered":"2024-04-17T06:09:20.634526+0000","last_unstale":"2024-04-17T06:09:25.659697+0000","last_undegraded":"2024-04-17T06:09:25.659697+0000","last_fullsized":"2024-04-17T06:09:25.659697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010538+0000","last_change":"2024-04-17T06:09:20.643072+0000","last_active":"2024-04-17T06:09:26.010538+0000","last_peered":"2024-04-17T06:09:26.010538+0000","last_clean":"2024-04-17T06:09:26.010538+0000","last_became_active":"2024-04-17T06:09:20.642812+0000","last_became_peered":"2024-04-17T06:09:20.642812+0000","last_unstale":"2024-04-17T06:09:26.010538+0000","last_undegraded":"2024-04-17T06:09:26.010538+0000","last_fullsized":"2024-04-17T06:09:26.010538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010750+0000","last_change":"2024-04-17T06:09:20.636421+0000","last_active":"2024-04-17T06:09:26.010750+0000","last_peered":"2024-04-17T06:09:26.010750+0000","last_clean":"2024-04-17T06:09:26.010750+0000","last_became_active":"2024-04-17T06:09:20.636165+0000","last_became_peered":"2024-04-17T06:09:20.636165+0000","last_unstale":"2024-04-17T06:09:26.010750+0000","last_undegraded":"2024-04-17T06:09:26.010750+0000","last_fullsized":"2024-04-17T06:09:26.010750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:41.556474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011251+0000","last_change":"2024-04-17T06:09:20.639331+0000","last_active":"2024-04-17T06:09:26.011251+0000","last_peered":"2024-04-17T06:09:26.011251+0000","last_clean":"2024-04-17T06:09:26.011251+0000","last_became_active":"2024-04-17T06:09:20.639051+0000","last_became_peered":"2024-04-17T06:09:20.639051+0000","last_unstale":"2024-04-17T06:09:26.011251+0000","last_undegraded":"2024-04-17T06:09:26.011251+0000","last_fullsized":"2024-04-17T06:09:26.011251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010813+0000","last_change":"2024-04-17T06:09:20.639239+0000","last_active":"2024-04-17T06:09:26.010813+0000","last_peered":"2024-04-17T06:09:26.010813+0000","last_clean":"2024-04-17T06:09:26.010813+0000","last_became_active":"2024-04-17T06:09:20.638982+0000","last_became_peered":"2024-04-17T06:09:20.638982+0000","last_unstale":"2024-04-17T06:09:26.010813+0000","last_undegraded":"2024-04-17T06:09:26.010813+0000","last_fullsized":"2024-04-17T06:09:26.010813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:08.892063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659939+0000","last_change":"2024-04-17T06:09:20.639508+0000","last_active":"2024-04-17T06:09:25.659939+0000","last_peered":"2024-04-17T06:09:25.659939+0000","last_clean":"2024-04-17T06:09:25.659939+0000","last_became_active":"2024-04-17T06:09:20.639151+0000","last_became_peered":"2024-04-17T06:09:20.639151+0000","last_unstale":"2024-04-17T06:09:25.659939+0000","last_undegraded":"2024-04-17T06:09:25.659939+0000","last_fullsized":"2024-04-17T06:09:25.659939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660890+0000","last_change":"2024-04-17T06:09:20.639084+0000","last_active":"2024-04-17T06:09:25.660890+0000","last_peered":"2024-04-17T06:09:25.660890+0000","last_clean":"2024-04-17T06:09:25.660890+0000","last_became_active":"2024-04-17T06:09:20.638793+0000","last_became_peered":"2024-04-17T06:09:20.638793+0000","last_unstale":"2024-04-17T06:09:25.660890+0000","last_undegraded":"2024-04-17T06:09:25.660890+0000","last_fullsized":"2024-04-17T06:09:25.660890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011592+0000","last_change":"2024-04-17T06:09:20.645741+0000","last_active":"2024-04-17T06:09:26.011592+0000","last_peered":"2024-04-17T06:09:26.011592+0000","last_clean":"2024-04-17T06:09:26.011592+0000","last_became_active":"2024-04-17T06:09:20.645168+0000","last_became_peered":"2024-04-17T06:09:20.645168+0000","last_unstale":"2024-04-17T06:09:26.011592+0000","last_undegraded":"2024-04-17T06:09:26.011592+0000","last_fullsized":"2024-04-17T06:09:26.011592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:16.143995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010798+0000","last_change":"2024-04-17T06:09:20.656219+0000","last_active":"2024-04-17T06:09:26.010798+0000","last_peered":"2024-04-17T06:09:26.010798+0000","last_clean":"2024-04-17T06:09:26.010798+0000","last_became_active":"2024-04-17T06:09:20.655978+0000","last_became_peered":"2024-04-17T06:09:20.655978+0000","last_unstale":"2024-04-17T06:09:26.010798+0000","last_undegraded":"2024-04-17T06:09:26.010798+0000","last_fullsized":"2024-04-17T06:09:26.010798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010761+0000","last_change":"2024-04-17T06:09:20.650090+0000","last_active":"2024-04-17T06:09:26.010761+0000","last_peered":"2024-04-17T06:09:26.010761+0000","last_clean":"2024-04-17T06:09:26.010761+0000","last_became_active":"2024-04-17T06:09:20.649596+0000","last_became_peered":"2024-04-17T06:09:20.649596+0000","last_unstale":"2024-04-17T06:09:26.010761+0000","last_undegraded":"2024-04-17T06:09:26.010761+0000","last_fullsized":"2024-04-17T06:09:26.010761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660177+0000","last_change":"2024-04-17T06:09:20.634204+0000","last_active":"2024-04-17T06:09:25.660177+0000","last_peered":"2024-04-17T06:09:25.660177+0000","last_clean":"2024-04-17T06:09:25.660177+0000","last_became_active":"2024-04-17T06:09:20.633924+0000","last_became_peered":"2024-04-17T06:09:20.633924+0000","last_unstale":"2024-04-17T06:09:25.660177+0000","last_undegraded":"2024-04-17T06:09:25.660177+0000","last_fullsized":"2024-04-17T06:09:25.660177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010796+0000","last_change":"2024-04-17T06:09:20.645437+0000","last_active":"2024-04-17T06:09:26.010796+0000","last_peered":"2024-04-17T06:09:26.010796+0000","last_clean":"2024-04-17T06:09:26.010796+0000","last_became_active":"2024-04-17T06:09:20.645015+0000","last_became_peered":"2024-04-17T06:09:20.645015+0000","last_unstale":"2024-04-17T06:09:26.010796+0000","last_undegraded":"2024-04-17T06:09:26.010796+0000","last_fullsized":"2024-04-17T06:09:26.010796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660367+0000","last_change":"2024-04-17T06:09:20.650436+0000","last_active":"2024-04-17T06:09:25.660367+0000","last_peered":"2024-04-17T06:09:25.660367+0000","last_clean":"2024-04-17T06:09:25.660367+0000","last_became_active":"2024-04-17T06:09:20.650132+0000","last_became_peered":"2024-04-17T06:09:20.650132+0000","last_unstale":"2024-04-17T06:09:25.660367+0000","last_undegraded":"2024-04-17T06:09:25.660367+0000","last_fullsized":"2024-04-17T06:09:25.660367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011167+0000","last_change":"2024-04-17T06:09:20.634255+0000","last_active":"2024-04-17T06:09:26.011167+0000","last_peered":"2024-04-17T06:09:26.011167+0000","last_clean":"2024-04-17T06:09:26.011167+0000","last_became_active":"2024-04-17T06:09:20.633879+0000","last_became_peered":"2024-04-17T06:09:20.633879+0000","last_unstale":"2024-04-17T06:09:26.011167+0000","last_undegraded":"2024-04-17T06:09:26.011167+0000","last_fullsized":"2024-04-17T06:09:26.011167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:07.365672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010938+0000","last_change":"2024-04-17T06:09:20.635023+0000","last_active":"2024-04-17T06:09:26.010938+0000","last_peered":"2024-04-17T06:09:26.010938+0000","last_clean":"2024-04-17T06:09:26.010938+0000","last_became_active":"2024-04-17T06:09:20.634245+0000","last_became_peered":"2024-04-17T06:09:20.634245+0000","last_unstale":"2024-04-17T06:09:26.010938+0000","last_undegraded":"2024-04-17T06:09:26.010938+0000","last_fullsized":"2024-04-17T06:09:26.010938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011506+0000","last_change":"2024-04-17T06:09:20.653636+0000","last_active":"2024-04-17T06:09:26.011506+0000","last_peered":"2024-04-17T06:09:26.011506+0000","last_clean":"2024-04-17T06:09:26.011506+0000","last_became_active":"2024-04-17T06:09:20.653366+0000","last_became_peered":"2024-04-17T06:09:20.653366+0000","last_unstale":"2024-04-17T06:09:26.011506+0000","last_undegraded":"2024-04-17T06:09:26.011506+0000","last_fullsized":"2024-04-17T06:09:26.011506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:58.610924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T06:09:37.679 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T06:09:37.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.817+0000 7fafa96f8640 1 Processor -- start 2024-04-17T06:09:37.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.817+0000 7fafa96f8640 1 -- start start 2024-04-17T06:09:37.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa96f8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fafa415ec00 0x7fafa40d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa96f8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fafa40d19e0 0x7fafa40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa96f8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa40d3330 con 0x7fafa40d43d0 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa96f8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fafa415efe0 con 0x7fafa40d19e0 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa96f8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fafa40d3f40 con 0x7fafa415ec00 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fafa40d19e0 0x7fafa40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fafa40d19e0 0x7fafa40b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48580/0 (socket says 172.21.15.135:48580) 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa37fe640 1 -- 172.21.15.135:0/567688887 learned_addr learned my addr 172.21.15.135:0/567688887 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa2ffd640 1 --2- 172.21.15.135:0/567688887 >> v2:172.21.15.190:3300/0 conn(0x7fafa415ec00 0x7fafa40d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa37fe640 1 -- 172.21.15.135:0/567688887 >> v2:172.21.15.190:3300/0 conn(0x7fafa415ec00 msgr2=0x7fafa40d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa37fe640 1 --2- 172.21.15.135:0/567688887 >> v2:172.21.15.190:3300/0 conn(0x7fafa415ec00 0x7fafa40d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa37fe640 1 -- 172.21.15.135:0/567688887 >> v1:172.21.15.190:6789/0 conn(0x7fafa40d43d0 legacy=0x7fafa40d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.818+0000 7fafa37fe640 1 -- 172.21.15.135:0/567688887 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa40d3610 con 0x7fafa40d19e0 2024-04-17T06:09:37.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.819+0000 7fafa37fe640 1 --2- 172.21.15.135:0/567688887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa40d19e0 0x7fafa40b7a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fafa406db40 tx=0x7faf9c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d7a4dc73b3dcea server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.819+0000 7fafa27fc640 1 -- 172.21.15.135:0/567688887 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c02cbb0 con 0x7fafa40d19e0 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.819+0000 7fafa27fc640 1 -- 172.21.15.135:0/567688887 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf9c02cd50 con 0x7fafa40d19e0 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.819+0000 7fafa27fc640 1 -- 172.21.15.135:0/567688887 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c034610 con 0x7fafa40d19e0 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.819+0000 7fafa96f8640 1 -- 172.21.15.135:0/567688887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa40d19e0 msgr2=0x7fafa40b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.819+0000 7fafa96f8640 1 --2- 172.21.15.135:0/567688887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa40d19e0 0x7fafa40b7a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fafa406db40 tx=0x7faf9c02c010 comp rx=0 tx=0).stop 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.820+0000 7fafa96f8640 1 -- 172.21.15.135:0/567688887 shutdown_connections 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.820+0000 7fafa96f8640 1 --2- 172.21.15.135:0/567688887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa40d19e0 0x7fafa40b7a40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.820+0000 7fafa96f8640 1 --2- 172.21.15.135:0/567688887 >> v2:172.21.15.190:3300/0 conn(0x7fafa415ec00 0x7fafa40d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.820+0000 7fafa96f8640 1 -- 172.21.15.135:0/567688887 >> 172.21.15.135:0/567688887 conn(0x7fafa406cc90 msgr2=0x7fafa406d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.820+0000 7fafa96f8640 1 -- 172.21.15.135:0/567688887 shutdown_connections 2024-04-17T06:09:37.819 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.820+0000 7fafa96f8640 1 -- 172.21.15.135:0/567688887 wait complete. 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.821+0000 7fafa96f8640 1 Processor -- start 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.821+0000 7fafa96f8640 1 -- start start 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa96f8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa40d19e0 0x7fafa40cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa96f8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa40d43d0 0x7fafa41506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa96f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa415ec00 0x7fafa40b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa96f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fafa40d2ed0 con 0x7fafa415ec00 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa96f8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fafa40d3bd0 con 0x7fafa40d19e0 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa96f8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fafa40d3050 con 0x7fafa40d43d0 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa415ec00 0x7fafa40b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa415ec00 0x7fafa40b91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48588/0 (socket says 172.21.15.135:48588) 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 -- 172.21.15.135:0/2666474553 learned_addr learned my addr 172.21.15.135:0/2666474553 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:37.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7faf9a7fc640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa40d43d0 0x7fafa41506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa2ffd640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa40d19e0 0x7fafa40cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 -- 172.21.15.135:0/2666474553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa40d43d0 msgr2=0x7fafa41506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa40d43d0 0x7fafa41506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 -- 172.21.15.135:0/2666474553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa40d19e0 msgr2=0x7fafa40cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa40d19e0 0x7fafa40cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 -- 172.21.15.135:0/2666474553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa41f8500 con 0x7fafa415ec00 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa37fe640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa415ec00 0x7fafa40b91a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009f40 tx=0x7faf9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7faf9bfff640 1 -- 172.21.15.135:0/2666474553 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c043020 con 0x7fafa415ec00 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7faf9bfff640 1 -- 172.21.15.135:0/2666474553 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf9c03e450 con 0x7fafa415ec00 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa41f87e0 con 0x7fafa415ec00 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.822+0000 7faf9bfff640 1 -- 172.21.15.135:0/2666474553 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c009510 con 0x7fafa415ec00 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.823+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa41f8da0 con 0x7fafa415ec00 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.824+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa4074940 con 0x7fafa415ec00 2024-04-17T06:09:37.823 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.824+0000 7faf9bfff640 1 -- 172.21.15.135:0/2666474553 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf9c046020 con 0x7fafa415ec00 2024-04-17T06:09:37.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.824+0000 7faf9bfff640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8406c8f0 0x7faf8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:37.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.824+0000 7faf9bfff640 1 -- 172.21.15.135:0/2666474553 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf9c0ae440 con 0x7fafa415ec00 2024-04-17T06:09:37.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.825+0000 7fafa2ffd640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8406c8f0 0x7faf8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:37.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.825+0000 7fafa2ffd640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8406c8f0 0x7faf8406edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fafa41601e0 tx=0x7faf94009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:37.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:37.829+0000 7faf9bfff640 1 -- 172.21.15.135:0/2666474553 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf9c04d030 con 0x7fafa415ec00 2024-04-17T06:09:37.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.001+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 --> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafa40d21e0 con 0x7faf8406c8f0 2024-04-17T06:09:38.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.009+0000 7faf9bfff640 1 -- 172.21.15.135:0/2666474553 <== mgr.4113 v2:172.21.15.135:6832/3190814219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fafa40d21e0 con 0x7faf8406c8f0 2024-04-17T06:09:38.010 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:38.012 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8406c8f0 msgr2=0x7faf8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8406c8f0 0x7faf8406edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fafa41601e0 tx=0x7faf94009190 comp rx=0 tx=0).stop 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa415ec00 msgr2=0x7fafa40b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa415ec00 0x7fafa40b91a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009f40 tx=0x7faf9c004a40 comp rx=0 tx=0).stop 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 shutdown_connections 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8406c8f0 0x7faf8406edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa415ec00 0x7fafa40b91a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa40d43d0 0x7fafa41506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 --2- 172.21.15.135:0/2666474553 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa40d19e0 0x7fafa40cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 >> 172.21.15.135:0/2666474553 conn(0x7fafa406cc90 msgr2=0x7fafa406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:38.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 shutdown_connections 2024-04-17T06:09:38.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.012+0000 7fafa96f8640 1 -- 172.21.15.135:0/2666474553 wait complete. 2024-04-17T06:09:38.021 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-04-17T06:09:37.941295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002255"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012356+0000","last_change":"2024-04-17T06:09:19.632439+0000","last_active":"2024-04-17T06:09:26.012356+0000","last_peered":"2024-04-17T06:09:26.012356+0000","last_clean":"2024-04-17T06:09:26.012356+0000","last_became_active":"2024-04-17T06:09:19.632170+0000","last_became_peered":"2024-04-17T06:09:19.632170+0000","last_unstale":"2024-04-17T06:09:26.012356+0000","last_undegraded":"2024-04-17T06:09:26.012356+0000","last_fullsized":"2024-04-17T06:09:26.012356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:58.547080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011716+0000","last_change":"2024-04-17T06:09:20.647290+0000","last_active":"2024-04-17T06:09:26.011716+0000","last_peered":"2024-04-17T06:09:26.011716+0000","last_clean":"2024-04-17T06:09:26.011716+0000","last_became_active":"2024-04-17T06:09:20.646950+0000","last_became_peered":"2024-04-17T06:09:20.646950+0000","last_unstale":"2024-04-17T06:09:26.011716+0000","last_undegraded":"2024-04-17T06:09:26.011716+0000","last_fullsized":"2024-04-17T06:09:26.011716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011791+0000","last_change":"2024-04-17T06:09:24.992961+0000","last_active":"2024-04-17T06:09:26.011791+0000","last_peered":"2024-04-17T06:09:26.011791+0000","last_clean":"2024-04-17T06:09:26.011791+0000","last_became_active":"2024-04-17T06:09:24.992540+0000","last_became_peered":"2024-04-17T06:09:24.992540+0000","last_unstale":"2024-04-17T06:09:26.011791+0000","last_undegraded":"2024-04-17T06:09:26.011791+0000","last_fullsized":"2024-04-17T06:09:26.011791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:42.943499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011930+0000","last_change":"2024-04-17T06:09:19.632406+0000","last_active":"2024-04-17T06:09:26.011930+0000","last_peered":"2024-04-17T06:09:26.011930+0000","last_clean":"2024-04-17T06:09:26.011930+0000","last_became_active":"2024-04-17T06:09:19.631990+0000","last_became_peered":"2024-04-17T06:09:19.631990+0000","last_unstale":"2024-04-17T06:09:26.011930+0000","last_undegraded":"2024-04-17T06:09:26.011930+0000","last_fullsized":"2024-04-17T06:09:26.011930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:35.753743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012065+0000","last_change":"2024-04-17T06:09:20.631799+0000","last_active":"2024-04-17T06:09:26.012065+0000","last_peered":"2024-04-17T06:09:26.012065+0000","last_clean":"2024-04-17T06:09:26.012065+0000","last_became_active":"2024-04-17T06:09:20.631551+0000","last_became_peered":"2024-04-17T06:09:20.631551+0000","last_unstale":"2024-04-17T06:09:26.012065+0000","last_undegraded":"2024-04-17T06:09:26.012065+0000","last_fullsized":"2024-04-17T06:09:26.012065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011413+0000","last_change":"2024-04-17T06:09:24.681573+0000","last_active":"2024-04-17T06:09:26.011413+0000","last_peered":"2024-04-17T06:09:26.011413+0000","last_clean":"2024-04-17T06:09:26.011413+0000","last_became_active":"2024-04-17T06:09:24.681194+0000","last_became_peered":"2024-04-17T06:09:24.681194+0000","last_unstale":"2024-04-17T06:09:26.011413+0000","last_undegraded":"2024-04-17T06:09:26.011413+0000","last_fullsized":"2024-04-17T06:09:26.011413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010848+0000","last_change":"2024-04-17T06:09:19.626563+0000","last_active":"2024-04-17T06:09:26.010848+0000","last_peered":"2024-04-17T06:09:26.010848+0000","last_clean":"2024-04-17T06:09:26.010848+0000","last_became_active":"2024-04-17T06:09:19.626169+0000","last_became_peered":"2024-04-17T06:09:19.626169+0000","last_unstale":"2024-04-17T06:09:26.010848+0000","last_undegraded":"2024-04-17T06:09:26.010848+0000","last_fullsized":"2024-04-17T06:09:26.010848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:07.389883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660478+0000","last_change":"2024-04-17T06:09:20.629675+0000","last_active":"2024-04-17T06:09:25.660478+0000","last_peered":"2024-04-17T06:09:25.660478+0000","last_clean":"2024-04-17T06:09:25.660478+0000","last_became_active":"2024-04-17T06:09:20.629335+0000","last_became_peered":"2024-04-17T06:09:20.629335+0000","last_unstale":"2024-04-17T06:09:25.660478+0000","last_undegraded":"2024-04-17T06:09:25.660478+0000","last_fullsized":"2024-04-17T06:09:25.660478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010974+0000","last_change":"2024-04-17T06:09:24.681484+0000","last_active":"2024-04-17T06:09:26.010974+0000","last_peered":"2024-04-17T06:09:26.010974+0000","last_clean":"2024-04-17T06:09:26.010974+0000","last_became_active":"2024-04-17T06:09:24.681074+0000","last_became_peered":"2024-04-17T06:09:24.681074+0000","last_unstale":"2024-04-17T06:09:26.010974+0000","last_undegraded":"2024-04-17T06:09:26.010974+0000","last_fullsized":"2024-04-17T06:09:26.010974+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:21.508074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011016+0000","last_change":"2024-04-17T06:09:19.629666+0000","last_active":"2024-04-17T06:09:26.011016+0000","last_peered":"2024-04-17T06:09:26.011016+0000","last_clean":"2024-04-17T06:09:26.011016+0000","last_became_active":"2024-04-17T06:09:19.629414+0000","last_became_peered":"2024-04-17T06:09:19.629414+0000","last_unstale":"2024-04-17T06:09:26.011016+0000","last_undegraded":"2024-04-17T06:09:26.011016+0000","last_fullsized":"2024-04-17T06:09:26.011016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660727+0000","last_change":"2024-04-17T06:09:20.629643+0000","last_active":"2024-04-17T06:09:25.660727+0000","last_peered":"2024-04-17T06:09:25.660727+0000","last_clean":"2024-04-17T06:09:25.660727+0000","last_became_active":"2024-04-17T06:09:20.629349+0000","last_became_peered":"2024-04-17T06:09:20.629349+0000","last_unstale":"2024-04-17T06:09:25.660727+0000","last_undegraded":"2024-04-17T06:09:25.660727+0000","last_fullsized":"2024-04-17T06:09:25.660727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:55.368933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011234+0000","last_change":"2024-04-17T06:09:24.673766+0000","last_active":"2024-04-17T06:09:26.011234+0000","last_peered":"2024-04-17T06:09:26.011234+0000","last_clean":"2024-04-17T06:09:26.011234+0000","last_became_active":"2024-04-17T06:09:24.673384+0000","last_became_peered":"2024-04-17T06:09:24.673384+0000","last_unstale":"2024-04-17T06:09:26.011234+0000","last_undegraded":"2024-04-17T06:09:26.011234+0000","last_fullsized":"2024-04-17T06:09:26.011234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:40.236016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659389+0000","last_change":"2024-04-17T06:09:20.643805+0000","last_active":"2024-04-17T06:09:25.659389+0000","last_peered":"2024-04-17T06:09:25.659389+0000","last_clean":"2024-04-17T06:09:25.659389+0000","last_became_active":"2024-04-17T06:09:20.643199+0000","last_became_peered":"2024-04-17T06:09:20.643199+0000","last_unstale":"2024-04-17T06:09:25.659389+0000","last_undegraded":"2024-04-17T06:09:25.659389+0000","last_fullsized":"2024-04-17T06:09:25.659389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:07.365672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011214+0000","last_change":"2024-04-17T06:09:19.626503+0000","last_active":"2024-04-17T06:09:26.011214+0000","last_peered":"2024-04-17T06:09:26.011214+0000","last_clean":"2024-04-17T06:09:26.011214+0000","last_became_active":"2024-04-17T06:09:19.625556+0000","last_became_peered":"2024-04-17T06:09:19.625556+0000","last_unstale":"2024-04-17T06:09:26.011214+0000","last_undegraded":"2024-04-17T06:09:26.011214+0000","last_fullsized":"2024-04-17T06:09:26.011214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010558+0000","last_change":"2024-04-17T06:09:24.680158+0000","last_active":"2024-04-17T06:09:26.010558+0000","last_peered":"2024-04-17T06:09:26.010558+0000","last_clean":"2024-04-17T06:09:26.010558+0000","last_became_active":"2024-04-17T06:09:24.679694+0000","last_became_peered":"2024-04-17T06:09:24.679694+0000","last_unstale":"2024-04-17T06:09:26.010558+0000","last_undegraded":"2024-04-17T06:09:26.010558+0000","last_fullsized":"2024-04-17T06:09:26.010558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.282988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011354+0000","last_change":"2024-04-17T06:09:20.638610+0000","last_active":"2024-04-17T06:09:26.011354+0000","last_peered":"2024-04-17T06:09:26.011354+0000","last_clean":"2024-04-17T06:09:26.011354+0000","last_became_active":"2024-04-17T06:09:20.638275+0000","last_became_peered":"2024-04-17T06:09:20.638275+0000","last_unstale":"2024-04-17T06:09:26.011354+0000","last_undegraded":"2024-04-17T06:09:26.011354+0000","last_fullsized":"2024-04-17T06:09:26.011354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011423+0000","last_change":"2024-04-17T06:09:19.626821+0000","last_active":"2024-04-17T06:09:26.011423+0000","last_peered":"2024-04-17T06:09:26.011423+0000","last_clean":"2024-04-17T06:09:26.011423+0000","last_became_active":"2024-04-17T06:09:19.626550+0000","last_became_peered":"2024-04-17T06:09:19.626550+0000","last_unstale":"2024-04-17T06:09:26.011423+0000","last_undegraded":"2024-04-17T06:09:26.011423+0000","last_fullsized":"2024-04-17T06:09:26.011423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010388+0000","last_change":"2024-04-17T06:09:24.680040+0000","last_active":"2024-04-17T06:09:26.010388+0000","last_peered":"2024-04-17T06:09:26.010388+0000","last_clean":"2024-04-17T06:09:26.010388+0000","last_became_active":"2024-04-17T06:09:24.679567+0000","last_became_peered":"2024-04-17T06:09:24.679567+0000","last_unstale":"2024-04-17T06:09:26.010388+0000","last_undegraded":"2024-04-17T06:09:26.010388+0000","last_fullsized":"2024-04-17T06:09:26.010388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:21.549944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010400+0000","last_change":"2024-04-17T06:09:20.645617+0000","last_active":"2024-04-17T06:09:26.010400+0000","last_peered":"2024-04-17T06:09:26.010400+0000","last_clean":"2024-04-17T06:09:26.010400+0000","last_became_active":"2024-04-17T06:09:20.645205+0000","last_became_peered":"2024-04-17T06:09:20.645205+0000","last_unstale":"2024-04-17T06:09:26.010400+0000","last_undegraded":"2024-04-17T06:09:26.010400+0000","last_fullsized":"2024-04-17T06:09:26.010400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010623+0000","last_change":"2024-04-17T06:09:19.623695+0000","last_active":"2024-04-17T06:09:26.010623+0000","last_peered":"2024-04-17T06:09:26.010623+0000","last_clean":"2024-04-17T06:09:26.010623+0000","last_became_active":"2024-04-17T06:09:19.623442+0000","last_became_peered":"2024-04-17T06:09:19.623442+0000","last_unstale":"2024-04-17T06:09:26.010623+0000","last_undegraded":"2024-04-17T06:09:26.010623+0000","last_fullsized":"2024-04-17T06:09:26.010623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:07.888042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010887+0000","last_change":"2024-04-17T06:09:24.990476+0000","last_active":"2024-04-17T06:09:26.010887+0000","last_peered":"2024-04-17T06:09:26.010887+0000","last_clean":"2024-04-17T06:09:26.010887+0000","last_became_active":"2024-04-17T06:09:24.990092+0000","last_became_peered":"2024-04-17T06:09:24.990092+0000","last_unstale":"2024-04-17T06:09:26.010887+0000","last_undegraded":"2024-04-17T06:09:26.010887+0000","last_fullsized":"2024-04-17T06:09:26.010887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010691+0000","last_change":"2024-04-17T06:09:20.633239+0000","last_active":"2024-04-17T06:09:26.010691+0000","last_peered":"2024-04-17T06:09:26.010691+0000","last_clean":"2024-04-17T06:09:26.010691+0000","last_became_active":"2024-04-17T06:09:20.632946+0000","last_became_peered":"2024-04-17T06:09:20.632946+0000","last_unstale":"2024-04-17T06:09:26.010691+0000","last_undegraded":"2024-04-17T06:09:26.010691+0000","last_fullsized":"2024-04-17T06:09:26.010691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:01.439932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010236+0000","last_change":"2024-04-17T06:09:19.632434+0000","last_active":"2024-04-17T06:09:26.010236+0000","last_peered":"2024-04-17T06:09:26.010236+0000","last_clean":"2024-04-17T06:09:26.010236+0000","last_became_active":"2024-04-17T06:09:19.632174+0000","last_became_peered":"2024-04-17T06:09:19.632174+0000","last_unstale":"2024-04-17T06:09:26.010236+0000","last_undegraded":"2024-04-17T06:09:26.010236+0000","last_fullsized":"2024-04-17T06:09:26.010236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:06.724008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010716+0000","last_change":"2024-04-17T06:09:24.680395+0000","last_active":"2024-04-17T06:09:26.010716+0000","last_peered":"2024-04-17T06:09:26.010716+0000","last_clean":"2024-04-17T06:09:26.010716+0000","last_became_active":"2024-04-17T06:09:24.679895+0000","last_became_peered":"2024-04-17T06:09:24.679895+0000","last_unstale":"2024-04-17T06:09:26.010716+0000","last_undegraded":"2024-04-17T06:09:26.010716+0000","last_fullsized":"2024-04-17T06:09:26.010716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:14.465883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012593+0000","last_change":"2024-04-17T06:09:20.638642+0000","last_active":"2024-04-17T06:09:26.012593+0000","last_peered":"2024-04-17T06:09:26.012593+0000","last_clean":"2024-04-17T06:09:26.012593+0000","last_became_active":"2024-04-17T06:09:20.638351+0000","last_became_peered":"2024-04-17T06:09:20.638351+0000","last_unstale":"2024-04-17T06:09:26.012593+0000","last_undegraded":"2024-04-17T06:09:26.012593+0000","last_fullsized":"2024-04-17T06:09:26.012593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660496+0000","last_change":"2024-04-17T06:09:19.622032+0000","last_active":"2024-04-17T06:09:25.660496+0000","last_peered":"2024-04-17T06:09:25.660496+0000","last_clean":"2024-04-17T06:09:25.660496+0000","last_became_active":"2024-04-17T06:09:19.621765+0000","last_became_peered":"2024-04-17T06:09:19.621765+0000","last_unstale":"2024-04-17T06:09:25.660496+0000","last_undegraded":"2024-04-17T06:09:25.660496+0000","last_fullsized":"2024-04-17T06:09:25.660496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010414+0000","last_change":"2024-04-17T06:09:24.990792+0000","last_active":"2024-04-17T06:09:26.010414+0000","last_peered":"2024-04-17T06:09:26.010414+0000","last_clean":"2024-04-17T06:09:26.010414+0000","last_became_active":"2024-04-17T06:09:24.990371+0000","last_became_peered":"2024-04-17T06:09:24.990371+0000","last_unstale":"2024-04-17T06:09:26.010414+0000","last_undegraded":"2024-04-17T06:09:26.010414+0000","last_fullsized":"2024-04-17T06:09:26.010414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:26.189133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011005+0000","last_change":"2024-04-17T06:09:20.648546+0000","last_active":"2024-04-17T06:09:26.011005+0000","last_peered":"2024-04-17T06:09:26.011005+0000","last_clean":"2024-04-17T06:09:26.011005+0000","last_became_active":"2024-04-17T06:09:20.647853+0000","last_became_peered":"2024-04-17T06:09:20.647853+0000","last_unstale":"2024-04-17T06:09:26.011005+0000","last_undegraded":"2024-04-17T06:09:26.011005+0000","last_fullsized":"2024-04-17T06:09:26.011005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011025+0000","last_change":"2024-04-17T06:09:19.627150+0000","last_active":"2024-04-17T06:09:26.011025+0000","last_peered":"2024-04-17T06:09:26.011025+0000","last_clean":"2024-04-17T06:09:26.011025+0000","last_became_active":"2024-04-17T06:09:19.626875+0000","last_became_peered":"2024-04-17T06:09:19.626875+0000","last_unstale":"2024-04-17T06:09:26.011025+0000","last_undegraded":"2024-04-17T06:09:26.011025+0000","last_fullsized":"2024-04-17T06:09:26.011025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010576+0000","last_change":"2024-04-17T06:09:24.992298+0000","last_active":"2024-04-17T06:09:26.010576+0000","last_peered":"2024-04-17T06:09:26.010576+0000","last_clean":"2024-04-17T06:09:26.010576+0000","last_became_active":"2024-04-17T06:09:24.991871+0000","last_became_peered":"2024-04-17T06:09:24.991871+0000","last_unstale":"2024-04-17T06:09:26.010576+0000","last_undegraded":"2024-04-17T06:09:26.010576+0000","last_fullsized":"2024-04-17T06:09:26.010576+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:43.835421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011016+0000","last_change":"2024-04-17T06:09:20.639206+0000","last_active":"2024-04-17T06:09:26.011016+0000","last_peered":"2024-04-17T06:09:26.011016+0000","last_clean":"2024-04-17T06:09:26.011016+0000","last_became_active":"2024-04-17T06:09:20.638954+0000","last_became_peered":"2024-04-17T06:09:20.638954+0000","last_unstale":"2024-04-17T06:09:26.011016+0000","last_undegraded":"2024-04-17T06:09:26.011016+0000","last_fullsized":"2024-04-17T06:09:26.011016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011535+0000","last_change":"2024-04-17T06:09:19.632347+0000","last_active":"2024-04-17T06:09:26.011535+0000","last_peered":"2024-04-17T06:09:26.011535+0000","last_clean":"2024-04-17T06:09:26.011535+0000","last_became_active":"2024-04-17T06:09:19.632117+0000","last_became_peered":"2024-04-17T06:09:19.632117+0000","last_unstale":"2024-04-17T06:09:26.011535+0000","last_undegraded":"2024-04-17T06:09:26.011535+0000","last_fullsized":"2024-04-17T06:09:26.011535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:07.888042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011384+0000","last_change":"2024-04-17T06:09:24.989732+0000","last_active":"2024-04-17T06:09:26.011384+0000","last_peered":"2024-04-17T06:09:26.011384+0000","last_clean":"2024-04-17T06:09:26.011384+0000","last_became_active":"2024-04-17T06:09:24.989320+0000","last_became_peered":"2024-04-17T06:09:24.989320+0000","last_unstale":"2024-04-17T06:09:26.011384+0000","last_undegraded":"2024-04-17T06:09:26.011384+0000","last_fullsized":"2024-04-17T06:09:26.011384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660737+0000","last_change":"2024-04-17T06:09:20.652167+0000","last_active":"2024-04-17T06:09:25.660737+0000","last_peered":"2024-04-17T06:09:25.660737+0000","last_clean":"2024-04-17T06:09:25.660737+0000","last_became_active":"2024-04-17T06:09:20.651908+0000","last_became_peered":"2024-04-17T06:09:20.651908+0000","last_unstale":"2024-04-17T06:09:25.660737+0000","last_undegraded":"2024-04-17T06:09:25.660737+0000","last_fullsized":"2024-04-17T06:09:25.660737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:58.610924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011096+0000","last_change":"2024-04-17T06:09:19.631750+0000","last_active":"2024-04-17T06:09:26.011096+0000","last_peered":"2024-04-17T06:09:26.011096+0000","last_clean":"2024-04-17T06:09:26.011096+0000","last_became_active":"2024-04-17T06:09:19.631463+0000","last_became_peered":"2024-04-17T06:09:19.631463+0000","last_unstale":"2024-04-17T06:09:26.011096+0000","last_undegraded":"2024-04-17T06:09:26.011096+0000","last_fullsized":"2024-04-17T06:09:26.011096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:07.389883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011105+0000","last_change":"2024-04-17T06:09:24.680787+0000","last_active":"2024-04-17T06:09:26.011105+0000","last_peered":"2024-04-17T06:09:26.011105+0000","last_clean":"2024-04-17T06:09:26.011105+0000","last_became_active":"2024-04-17T06:09:24.680279+0000","last_became_peered":"2024-04-17T06:09:24.680279+0000","last_unstale":"2024-04-17T06:09:26.011105+0000","last_undegraded":"2024-04-17T06:09:26.011105+0000","last_fullsized":"2024-04-17T06:09:26.011105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:25.702449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011262+0000","last_change":"2024-04-17T06:09:20.632789+0000","last_active":"2024-04-17T06:09:26.011262+0000","last_peered":"2024-04-17T06:09:26.011262+0000","last_clean":"2024-04-17T06:09:26.011262+0000","last_became_active":"2024-04-17T06:09:20.632199+0000","last_became_peered":"2024-04-17T06:09:20.632199+0000","last_unstale":"2024-04-17T06:09:26.011262+0000","last_undegraded":"2024-04-17T06:09:26.011262+0000","last_fullsized":"2024-04-17T06:09:26.011262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:08.393904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660530+0000","last_change":"2024-04-17T06:09:19.624615+0000","last_active":"2024-04-17T06:09:25.660530+0000","last_peered":"2024-04-17T06:09:25.660530+0000","last_clean":"2024-04-17T06:09:25.660530+0000","last_became_active":"2024-04-17T06:09:19.624333+0000","last_became_peered":"2024-04-17T06:09:19.624333+0000","last_unstale":"2024-04-17T06:09:25.660530+0000","last_undegraded":"2024-04-17T06:09:25.660530+0000","last_fullsized":"2024-04-17T06:09:25.660530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011059+0000","last_change":"2024-04-17T06:09:24.675868+0000","last_active":"2024-04-17T06:09:26.011059+0000","last_peered":"2024-04-17T06:09:26.011059+0000","last_clean":"2024-04-17T06:09:26.011059+0000","last_became_active":"2024-04-17T06:09:24.675269+0000","last_became_peered":"2024-04-17T06:09:24.675269+0000","last_unstale":"2024-04-17T06:09:26.011059+0000","last_undegraded":"2024-04-17T06:09:26.011059+0000","last_fullsized":"2024-04-17T06:09:26.011059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:02.650317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010687+0000","last_change":"2024-04-17T06:09:20.638626+0000","last_active":"2024-04-17T06:09:26.010687+0000","last_peered":"2024-04-17T06:09:26.010687+0000","last_clean":"2024-04-17T06:09:26.010687+0000","last_became_active":"2024-04-17T06:09:20.638339+0000","last_became_peered":"2024-04-17T06:09:20.638339+0000","last_unstale":"2024-04-17T06:09:26.010687+0000","last_undegraded":"2024-04-17T06:09:26.010687+0000","last_fullsized":"2024-04-17T06:09:26.010687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661071+0000","last_change":"2024-04-17T06:09:19.621292+0000","last_active":"2024-04-17T06:09:25.661071+0000","last_peered":"2024-04-17T06:09:25.661071+0000","last_clean":"2024-04-17T06:09:25.661071+0000","last_became_active":"2024-04-17T06:09:19.620827+0000","last_became_peered":"2024-04-17T06:09:19.620827+0000","last_unstale":"2024-04-17T06:09:25.661071+0000","last_undegraded":"2024-04-17T06:09:25.661071+0000","last_fullsized":"2024-04-17T06:09:25.661071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011064+0000","last_change":"2024-04-17T06:09:24.682574+0000","last_active":"2024-04-17T06:09:26.011064+0000","last_peered":"2024-04-17T06:09:26.011064+0000","last_clean":"2024-04-17T06:09:26.011064+0000","last_became_active":"2024-04-17T06:09:24.680550+0000","last_became_peered":"2024-04-17T06:09:24.680550+0000","last_unstale":"2024-04-17T06:09:26.011064+0000","last_undegraded":"2024-04-17T06:09:26.011064+0000","last_fullsized":"2024-04-17T06:09:26.011064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:20.183387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010301+0000","last_change":"2024-04-17T06:09:20.630308+0000","last_active":"2024-04-17T06:09:26.010301+0000","last_peered":"2024-04-17T06:09:26.010301+0000","last_clean":"2024-04-17T06:09:26.010301+0000","last_became_active":"2024-04-17T06:09:20.630052+0000","last_became_peered":"2024-04-17T06:09:20.630052+0000","last_unstale":"2024-04-17T06:09:26.010301+0000","last_undegraded":"2024-04-17T06:09:26.010301+0000","last_fullsized":"2024-04-17T06:09:26.010301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:53.394999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010349+0000","last_change":"2024-04-17T06:09:19.624643+0000","last_active":"2024-04-17T06:09:26.010349+0000","last_peered":"2024-04-17T06:09:26.010349+0000","last_clean":"2024-04-17T06:09:26.010349+0000","last_became_active":"2024-04-17T06:09:19.624270+0000","last_became_peered":"2024-04-17T06:09:19.624270+0000","last_unstale":"2024-04-17T06:09:26.010349+0000","last_undegraded":"2024-04-17T06:09:26.010349+0000","last_fullsized":"2024-04-17T06:09:26.010349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012635+0000","last_change":"2024-04-17T06:09:24.989917+0000","last_active":"2024-04-17T06:09:26.012635+0000","last_peered":"2024-04-17T06:09:26.012635+0000","last_clean":"2024-04-17T06:09:26.012635+0000","last_became_active":"2024-04-17T06:09:24.989308+0000","last_became_peered":"2024-04-17T06:09:24.989308+0000","last_unstale":"2024-04-17T06:09:26.012635+0000","last_undegraded":"2024-04-17T06:09:26.012635+0000","last_fullsized":"2024-04-17T06:09:26.012635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:14:19.804372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010303+0000","last_change":"2024-04-17T06:09:20.656161+0000","last_active":"2024-04-17T06:09:26.010303+0000","last_peered":"2024-04-17T06:09:26.010303+0000","last_clean":"2024-04-17T06:09:26.010303+0000","last_became_active":"2024-04-17T06:09:20.655897+0000","last_became_peered":"2024-04-17T06:09:20.655897+0000","last_unstale":"2024-04-17T06:09:26.010303+0000","last_undegraded":"2024-04-17T06:09:26.010303+0000","last_fullsized":"2024-04-17T06:09:26.010303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660641+0000","last_change":"2024-04-17T06:09:19.621223+0000","last_active":"2024-04-17T06:09:25.660641+0000","last_peered":"2024-04-17T06:09:25.660641+0000","last_clean":"2024-04-17T06:09:25.660641+0000","last_became_active":"2024-04-17T06:09:19.620409+0000","last_became_peered":"2024-04-17T06:09:19.620409+0000","last_unstale":"2024-04-17T06:09:25.660641+0000","last_undegraded":"2024-04-17T06:09:25.660641+0000","last_fullsized":"2024-04-17T06:09:25.660641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010841+0000","last_change":"2024-04-17T06:09:09.067035+0000","last_active":"2024-04-17T06:09:26.010841+0000","last_peered":"2024-04-17T06:09:26.010841+0000","last_clean":"2024-04-17T06:09:26.010841+0000","last_became_active":"2024-04-17T06:09:09.066130+0000","last_became_peered":"2024-04-17T06:09:09.066130+0000","last_unstale":"2024-04-17T06:09:26.010841+0000","last_undegraded":"2024-04-17T06:09:26.010841+0000","last_fullsized":"2024-04-17T06:09:26.010841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_clean_scrub_stamp":"2024-04-17T06:09:07.958875+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:03.277836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659861+0000","last_change":"2024-04-17T06:09:24.992008+0000","last_active":"2024-04-17T06:09:25.659861+0000","last_peered":"2024-04-17T06:09:25.659861+0000","last_clean":"2024-04-17T06:09:25.659861+0000","last_became_active":"2024-04-17T06:09:24.991448+0000","last_became_peered":"2024-04-17T06:09:24.991448+0000","last_unstale":"2024-04-17T06:09:25.659861+0000","last_undegraded":"2024-04-17T06:09:25.659861+0000","last_fullsized":"2024-04-17T06:09:25.659861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:15.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010640+0000","last_change":"2024-04-17T06:09:20.646916+0000","last_active":"2024-04-17T06:09:26.010640+0000","last_peered":"2024-04-17T06:09:26.010640+0000","last_clean":"2024-04-17T06:09:26.010640+0000","last_became_active":"2024-04-17T06:09:20.646069+0000","last_became_peered":"2024-04-17T06:09:20.646069+0000","last_unstale":"2024-04-17T06:09:26.010640+0000","last_undegraded":"2024-04-17T06:09:26.010640+0000","last_fullsized":"2024-04-17T06:09:26.010640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:11.442995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661446+0000","last_change":"2024-04-17T06:09:19.626825+0000","last_active":"2024-04-17T06:09:25.661446+0000","last_peered":"2024-04-17T06:09:25.661446+0000","last_clean":"2024-04-17T06:09:25.661446+0000","last_became_active":"2024-04-17T06:09:19.626549+0000","last_became_peered":"2024-04-17T06:09:19.626549+0000","last_unstale":"2024-04-17T06:09:25.661446+0000","last_undegraded":"2024-04-17T06:09:25.661446+0000","last_fullsized":"2024-04-17T06:09:25.661446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659329+0000","last_change":"2024-04-17T06:09:24.990665+0000","last_active":"2024-04-17T06:09:25.659329+0000","last_peered":"2024-04-17T06:09:25.659329+0000","last_clean":"2024-04-17T06:09:25.659329+0000","last_became_active":"2024-04-17T06:09:24.990262+0000","last_became_peered":"2024-04-17T06:09:24.990262+0000","last_unstale":"2024-04-17T06:09:25.659329+0000","last_undegraded":"2024-04-17T06:09:25.659329+0000","last_fullsized":"2024-04-17T06:09:25.659329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:21.508074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660652+0000","last_change":"2024-04-17T06:09:20.650264+0000","last_active":"2024-04-17T06:09:25.660652+0000","last_peered":"2024-04-17T06:09:25.660652+0000","last_clean":"2024-04-17T06:09:25.660652+0000","last_became_active":"2024-04-17T06:09:20.649792+0000","last_became_peered":"2024-04-17T06:09:20.649792+0000","last_unstale":"2024-04-17T06:09:25.660652+0000","last_undegraded":"2024-04-17T06:09:25.660652+0000","last_fullsized":"2024-04-17T06:09:25.660652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:36.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011681+0000","last_change":"2024-04-17T06:09:19.625656+0000","last_active":"2024-04-17T06:09:26.011681+0000","last_peered":"2024-04-17T06:09:26.011681+0000","last_clean":"2024-04-17T06:09:26.011681+0000","last_became_active":"2024-04-17T06:09:19.625408+0000","last_became_peered":"2024-04-17T06:09:19.625408+0000","last_unstale":"2024-04-17T06:09:26.011681+0000","last_undegraded":"2024-04-17T06:09:26.011681+0000","last_fullsized":"2024-04-17T06:09:26.011681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011611+0000","last_change":"2024-04-17T06:09:24.684050+0000","last_active":"2024-04-17T06:09:26.011611+0000","last_peered":"2024-04-17T06:09:26.011611+0000","last_clean":"2024-04-17T06:09:26.011611+0000","last_became_active":"2024-04-17T06:09:24.683688+0000","last_became_peered":"2024-04-17T06:09:24.683688+0000","last_unstale":"2024-04-17T06:09:26.011611+0000","last_undegraded":"2024-04-17T06:09:26.011611+0000","last_fullsized":"2024-04-17T06:09:26.011611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:02.650317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011613+0000","last_change":"2024-04-17T06:09:20.645936+0000","last_active":"2024-04-17T06:09:26.011613+0000","last_peered":"2024-04-17T06:09:26.011613+0000","last_clean":"2024-04-17T06:09:26.011613+0000","last_became_active":"2024-04-17T06:09:20.645453+0000","last_became_peered":"2024-04-17T06:09:20.645453+0000","last_unstale":"2024-04-17T06:09:26.011613+0000","last_undegraded":"2024-04-17T06:09:26.011613+0000","last_fullsized":"2024-04-17T06:09:26.011613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:21.663057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661396+0000","last_change":"2024-04-17T06:09:19.621217+0000","last_active":"2024-04-17T06:09:25.661396+0000","last_peered":"2024-04-17T06:09:25.661396+0000","last_clean":"2024-04-17T06:09:25.661396+0000","last_became_active":"2024-04-17T06:09:19.620846+0000","last_became_peered":"2024-04-17T06:09:19.620846+0000","last_unstale":"2024-04-17T06:09:25.661396+0000","last_undegraded":"2024-04-17T06:09:25.661396+0000","last_fullsized":"2024-04-17T06:09:25.661396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659685+0000","last_change":"2024-04-17T06:09:24.991134+0000","last_active":"2024-04-17T06:09:25.659685+0000","last_peered":"2024-04-17T06:09:25.659685+0000","last_clean":"2024-04-17T06:09:25.659685+0000","last_became_active":"2024-04-17T06:09:24.990742+0000","last_became_peered":"2024-04-17T06:09:24.990742+0000","last_unstale":"2024-04-17T06:09:25.659685+0000","last_undegraded":"2024-04-17T06:09:25.659685+0000","last_fullsized":"2024-04-17T06:09:25.659685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:54.153325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011276+0000","last_change":"2024-04-17T06:09:20.645660+0000","last_active":"2024-04-17T06:09:26.011276+0000","last_peered":"2024-04-17T06:09:26.011276+0000","last_clean":"2024-04-17T06:09:26.011276+0000","last_became_active":"2024-04-17T06:09:20.645370+0000","last_became_peered":"2024-04-17T06:09:20.645370+0000","last_unstale":"2024-04-17T06:09:26.011276+0000","last_undegraded":"2024-04-17T06:09:26.011276+0000","last_fullsized":"2024-04-17T06:09:26.011276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:36.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011721+0000","last_change":"2024-04-17T06:09:19.633064+0000","last_active":"2024-04-17T06:09:26.011721+0000","last_peered":"2024-04-17T06:09:26.011721+0000","last_clean":"2024-04-17T06:09:26.011721+0000","last_became_active":"2024-04-17T06:09:19.632759+0000","last_became_peered":"2024-04-17T06:09:19.632759+0000","last_unstale":"2024-04-17T06:09:26.011721+0000","last_undegraded":"2024-04-17T06:09:26.011721+0000","last_fullsized":"2024-04-17T06:09:26.011721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:58.547080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010213+0000","last_change":"2024-04-17T06:09:24.676408+0000","last_active":"2024-04-17T06:09:26.010213+0000","last_peered":"2024-04-17T06:09:26.010213+0000","last_clean":"2024-04-17T06:09:26.010213+0000","last_became_active":"2024-04-17T06:09:24.675996+0000","last_became_peered":"2024-04-17T06:09:24.675996+0000","last_unstale":"2024-04-17T06:09:26.010213+0000","last_undegraded":"2024-04-17T06:09:26.010213+0000","last_fullsized":"2024-04-17T06:09:26.010213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:14.169938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010896+0000","last_change":"2024-04-17T06:09:20.645388+0000","last_active":"2024-04-17T06:09:26.010896+0000","last_peered":"2024-04-17T06:09:26.010896+0000","last_clean":"2024-04-17T06:09:26.010896+0000","last_became_active":"2024-04-17T06:09:20.645091+0000","last_became_peered":"2024-04-17T06:09:20.645091+0000","last_unstale":"2024-04-17T06:09:26.010896+0000","last_undegraded":"2024-04-17T06:09:26.010896+0000","last_fullsized":"2024-04-17T06:09:26.010896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660949+0000","last_change":"2024-04-17T06:09:19.624671+0000","last_active":"2024-04-17T06:09:25.660949+0000","last_peered":"2024-04-17T06:09:25.660949+0000","last_clean":"2024-04-17T06:09:25.660949+0000","last_became_active":"2024-04-17T06:09:19.624360+0000","last_became_peered":"2024-04-17T06:09:19.624360+0000","last_unstale":"2024-04-17T06:09:25.660949+0000","last_undegraded":"2024-04-17T06:09:25.660949+0000","last_fullsized":"2024-04-17T06:09:25.660949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:35.753743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012475+0000","last_change":"2024-04-17T06:09:24.677224+0000","last_active":"2024-04-17T06:09:26.012475+0000","last_peered":"2024-04-17T06:09:26.012475+0000","last_clean":"2024-04-17T06:09:26.012475+0000","last_became_active":"2024-04-17T06:09:24.676834+0000","last_became_peered":"2024-04-17T06:09:24.676834+0000","last_unstale":"2024-04-17T06:09:26.012475+0000","last_undegraded":"2024-04-17T06:09:26.012475+0000","last_fullsized":"2024-04-17T06:09:26.012475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:25.702449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659412+0000","last_change":"2024-04-17T06:09:20.640720+0000","last_active":"2024-04-17T06:09:25.659412+0000","last_peered":"2024-04-17T06:09:25.659412+0000","last_clean":"2024-04-17T06:09:25.659412+0000","last_became_active":"2024-04-17T06:09:20.639180+0000","last_became_peered":"2024-04-17T06:09:20.639180+0000","last_unstale":"2024-04-17T06:09:25.659412+0000","last_undegraded":"2024-04-17T06:09:25.659412+0000","last_fullsized":"2024-04-17T06:09:25.659412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659510+0000","last_change":"2024-04-17T06:09:19.623937+0000","last_active":"2024-04-17T06:09:25.659510+0000","last_peered":"2024-04-17T06:09:25.659510+0000","last_clean":"2024-04-17T06:09:25.659510+0000","last_became_active":"2024-04-17T06:09:19.623613+0000","last_became_peered":"2024-04-17T06:09:19.623613+0000","last_unstale":"2024-04-17T06:09:25.659510+0000","last_undegraded":"2024-04-17T06:09:25.659510+0000","last_fullsized":"2024-04-17T06:09:25.659510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010862+0000","last_change":"2024-04-17T06:09:24.991659+0000","last_active":"2024-04-17T06:09:26.010862+0000","last_peered":"2024-04-17T06:09:26.010862+0000","last_clean":"2024-04-17T06:09:26.010862+0000","last_became_active":"2024-04-17T06:09:24.991305+0000","last_became_peered":"2024-04-17T06:09:24.991305+0000","last_unstale":"2024-04-17T06:09:26.010862+0000","last_undegraded":"2024-04-17T06:09:26.010862+0000","last_fullsized":"2024-04-17T06:09:26.010862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:22.763944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010997+0000","last_change":"2024-04-17T06:09:20.647491+0000","last_active":"2024-04-17T06:09:26.010997+0000","last_peered":"2024-04-17T06:09:26.010997+0000","last_clean":"2024-04-17T06:09:26.010997+0000","last_became_active":"2024-04-17T06:09:20.647222+0000","last_became_peered":"2024-04-17T06:09:20.647222+0000","last_unstale":"2024-04-17T06:09:26.010997+0000","last_undegraded":"2024-04-17T06:09:26.010997+0000","last_fullsized":"2024-04-17T06:09:26.010997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660425+0000","last_change":"2024-04-17T06:09:19.622734+0000","last_active":"2024-04-17T06:09:25.660425+0000","last_peered":"2024-04-17T06:09:25.660425+0000","last_clean":"2024-04-17T06:09:25.660425+0000","last_became_active":"2024-04-17T06:09:19.622401+0000","last_became_peered":"2024-04-17T06:09:19.622401+0000","last_unstale":"2024-04-17T06:09:25.660425+0000","last_undegraded":"2024-04-17T06:09:25.660425+0000","last_fullsized":"2024-04-17T06:09:25.660425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012343+0000","last_change":"2024-04-17T06:09:24.989713+0000","last_active":"2024-04-17T06:09:26.012343+0000","last_peered":"2024-04-17T06:09:26.012343+0000","last_clean":"2024-04-17T06:09:26.012343+0000","last_became_active":"2024-04-17T06:09:24.989311+0000","last_became_peered":"2024-04-17T06:09:24.989311+0000","last_unstale":"2024-04-17T06:09:26.012343+0000","last_undegraded":"2024-04-17T06:09:26.012343+0000","last_fullsized":"2024-04-17T06:09:26.012343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:09.035921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010838+0000","last_change":"2024-04-17T06:09:20.648156+0000","last_active":"2024-04-17T06:09:26.010838+0000","last_peered":"2024-04-17T06:09:26.010838+0000","last_clean":"2024-04-17T06:09:26.010838+0000","last_became_active":"2024-04-17T06:09:20.647878+0000","last_became_peered":"2024-04-17T06:09:20.647878+0000","last_unstale":"2024-04-17T06:09:26.010838+0000","last_undegraded":"2024-04-17T06:09:26.010838+0000","last_fullsized":"2024-04-17T06:09:26.010838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011779+0000","last_change":"2024-04-17T06:09:19.631943+0000","last_active":"2024-04-17T06:09:26.011779+0000","last_peered":"2024-04-17T06:09:26.011779+0000","last_clean":"2024-04-17T06:09:26.011779+0000","last_became_active":"2024-04-17T06:09:19.631287+0000","last_became_peered":"2024-04-17T06:09:19.631287+0000","last_unstale":"2024-04-17T06:09:26.011779+0000","last_undegraded":"2024-04-17T06:09:26.011779+0000","last_fullsized":"2024-04-17T06:09:26.011779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:44.893832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660591+0000","last_change":"2024-04-17T06:09:24.989665+0000","last_active":"2024-04-17T06:09:25.660591+0000","last_peered":"2024-04-17T06:09:25.660591+0000","last_clean":"2024-04-17T06:09:25.660591+0000","last_became_active":"2024-04-17T06:09:24.989241+0000","last_became_peered":"2024-04-17T06:09:24.989241+0000","last_unstale":"2024-04-17T06:09:25.660591+0000","last_undegraded":"2024-04-17T06:09:25.660591+0000","last_fullsized":"2024-04-17T06:09:25.660591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:43.835421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010532+0000","last_change":"2024-04-17T06:09:20.647147+0000","last_active":"2024-04-17T06:09:26.010532+0000","last_peered":"2024-04-17T06:09:26.010532+0000","last_clean":"2024-04-17T06:09:26.010532+0000","last_became_active":"2024-04-17T06:09:20.646657+0000","last_became_peered":"2024-04-17T06:09:20.646657+0000","last_unstale":"2024-04-17T06:09:26.010532+0000","last_undegraded":"2024-04-17T06:09:26.010532+0000","last_fullsized":"2024-04-17T06:09:26.010532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:30.115357+0000","last_change":"2024-04-17T06:09:19.629001+0000","last_active":"2024-04-17T06:09:30.115357+0000","last_peered":"2024-04-17T06:09:30.115357+0000","last_clean":"2024-04-17T06:09:30.115357+0000","last_became_active":"2024-04-17T06:09:19.628654+0000","last_became_peered":"2024-04-17T06:09:19.628654+0000","last_unstale":"2024-04-17T06:09:30.115357+0000","last_undegraded":"2024-04-17T06:09:30.115357+0000","last_fullsized":"2024-04-17T06:09:30.115357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011319+0000","last_change":"2024-04-17T06:09:24.682555+0000","last_active":"2024-04-17T06:09:26.011319+0000","last_peered":"2024-04-17T06:09:26.011319+0000","last_clean":"2024-04-17T06:09:26.011319+0000","last_became_active":"2024-04-17T06:09:24.680576+0000","last_became_peered":"2024-04-17T06:09:24.680576+0000","last_unstale":"2024-04-17T06:09:26.011319+0000","last_undegraded":"2024-04-17T06:09:26.011319+0000","last_fullsized":"2024-04-17T06:09:26.011319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:14:19.804372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010916+0000","last_change":"2024-04-17T06:09:20.644222+0000","last_active":"2024-04-17T06:09:26.010916+0000","last_peered":"2024-04-17T06:09:26.010916+0000","last_clean":"2024-04-17T06:09:26.010916+0000","last_became_active":"2024-04-17T06:09:20.643862+0000","last_became_peered":"2024-04-17T06:09:20.643862+0000","last_unstale":"2024-04-17T06:09:26.010916+0000","last_undegraded":"2024-04-17T06:09:26.010916+0000","last_fullsized":"2024-04-17T06:09:26.010916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:07.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011583+0000","last_change":"2024-04-17T06:09:19.628512+0000","last_active":"2024-04-17T06:09:26.011583+0000","last_peered":"2024-04-17T06:09:26.011583+0000","last_clean":"2024-04-17T06:09:26.011583+0000","last_became_active":"2024-04-17T06:09:19.628227+0000","last_became_peered":"2024-04-17T06:09:19.628227+0000","last_unstale":"2024-04-17T06:09:26.011583+0000","last_undegraded":"2024-04-17T06:09:26.011583+0000","last_fullsized":"2024-04-17T06:09:26.011583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011193+0000","last_change":"2024-04-17T06:09:24.682017+0000","last_active":"2024-04-17T06:09:26.011193+0000","last_peered":"2024-04-17T06:09:26.011193+0000","last_clean":"2024-04-17T06:09:26.011193+0000","last_became_active":"2024-04-17T06:09:24.681637+0000","last_became_peered":"2024-04-17T06:09:24.681637+0000","last_unstale":"2024-04-17T06:09:26.011193+0000","last_undegraded":"2024-04-17T06:09:26.011193+0000","last_fullsized":"2024-04-17T06:09:26.011193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:15.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012238+0000","last_change":"2024-04-17T06:09:20.638985+0000","last_active":"2024-04-17T06:09:26.012238+0000","last_peered":"2024-04-17T06:09:26.012238+0000","last_clean":"2024-04-17T06:09:26.012238+0000","last_became_active":"2024-04-17T06:09:20.638132+0000","last_became_peered":"2024-04-17T06:09:20.638132+0000","last_unstale":"2024-04-17T06:09:26.012238+0000","last_undegraded":"2024-04-17T06:09:26.012238+0000","last_fullsized":"2024-04-17T06:09:26.012238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660523+0000","last_change":"2024-04-17T06:09:19.621789+0000","last_active":"2024-04-17T06:09:25.660523+0000","last_peered":"2024-04-17T06:09:25.660523+0000","last_clean":"2024-04-17T06:09:25.660523+0000","last_became_active":"2024-04-17T06:09:19.621506+0000","last_became_peered":"2024-04-17T06:09:19.621506+0000","last_unstale":"2024-04-17T06:09:25.660523+0000","last_undegraded":"2024-04-17T06:09:25.660523+0000","last_fullsized":"2024-04-17T06:09:25.660523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010264+0000","last_change":"2024-04-17T06:09:24.676641+0000","last_active":"2024-04-17T06:09:26.010264+0000","last_peered":"2024-04-17T06:09:26.010264+0000","last_clean":"2024-04-17T06:09:26.010264+0000","last_became_active":"2024-04-17T06:09:24.676251+0000","last_became_peered":"2024-04-17T06:09:24.676251+0000","last_unstale":"2024-04-17T06:09:26.010264+0000","last_undegraded":"2024-04-17T06:09:26.010264+0000","last_fullsized":"2024-04-17T06:09:26.010264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:54.153325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660474+0000","last_change":"2024-04-17T06:09:20.643983+0000","last_active":"2024-04-17T06:09:25.660474+0000","last_peered":"2024-04-17T06:09:25.660474+0000","last_clean":"2024-04-17T06:09:25.660474+0000","last_became_active":"2024-04-17T06:09:20.643704+0000","last_became_peered":"2024-04-17T06:09:20.643704+0000","last_unstale":"2024-04-17T06:09:25.660474+0000","last_undegraded":"2024-04-17T06:09:25.660474+0000","last_fullsized":"2024-04-17T06:09:25.660474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660308+0000","last_change":"2024-04-17T06:09:19.626344+0000","last_active":"2024-04-17T06:09:25.660308+0000","last_peered":"2024-04-17T06:09:25.660308+0000","last_clean":"2024-04-17T06:09:25.660308+0000","last_became_active":"2024-04-17T06:09:19.626060+0000","last_became_peered":"2024-04-17T06:09:19.626060+0000","last_unstale":"2024-04-17T06:09:25.660308+0000","last_undegraded":"2024-04-17T06:09:25.660308+0000","last_fullsized":"2024-04-17T06:09:25.660308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011948+0000","last_change":"2024-04-17T06:09:24.990709+0000","last_active":"2024-04-17T06:09:26.011948+0000","last_peered":"2024-04-17T06:09:26.011948+0000","last_clean":"2024-04-17T06:09:26.011948+0000","last_became_active":"2024-04-17T06:09:24.990327+0000","last_became_peered":"2024-04-17T06:09:24.990327+0000","last_unstale":"2024-04-17T06:09:26.011948+0000","last_undegraded":"2024-04-17T06:09:26.011948+0000","last_fullsized":"2024-04-17T06:09:26.011948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:26:33.123735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012474+0000","last_change":"2024-04-17T06:09:24.994579+0000","last_active":"2024-04-17T06:09:26.012474+0000","last_peered":"2024-04-17T06:09:26.012474+0000","last_clean":"2024-04-17T06:09:26.012474+0000","last_became_active":"2024-04-17T06:09:24.994172+0000","last_became_peered":"2024-04-17T06:09:24.994172+0000","last_unstale":"2024-04-17T06:09:26.012474+0000","last_undegraded":"2024-04-17T06:09:26.012474+0000","last_fullsized":"2024-04-17T06:09:26.012474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:20.183387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010445+0000","last_change":"2024-04-17T06:09:20.646244+0000","last_active":"2024-04-17T06:09:26.010445+0000","last_peered":"2024-04-17T06:09:26.010445+0000","last_clean":"2024-04-17T06:09:26.010445+0000","last_became_active":"2024-04-17T06:09:20.645899+0000","last_became_peered":"2024-04-17T06:09:20.645899+0000","last_unstale":"2024-04-17T06:09:26.010445+0000","last_undegraded":"2024-04-17T06:09:26.010445+0000","last_fullsized":"2024-04-17T06:09:26.010445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659810+0000","last_change":"2024-04-17T06:09:19.621004+0000","last_active":"2024-04-17T06:09:25.659810+0000","last_peered":"2024-04-17T06:09:25.659810+0000","last_clean":"2024-04-17T06:09:25.659810+0000","last_became_active":"2024-04-17T06:09:19.620503+0000","last_became_peered":"2024-04-17T06:09:19.620503+0000","last_unstale":"2024-04-17T06:09:25.659810+0000","last_undegraded":"2024-04-17T06:09:25.659810+0000","last_fullsized":"2024-04-17T06:09:25.659810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660166+0000","last_change":"2024-04-17T06:09:24.674536+0000","last_active":"2024-04-17T06:09:25.660166+0000","last_peered":"2024-04-17T06:09:25.660166+0000","last_clean":"2024-04-17T06:09:25.660166+0000","last_became_active":"2024-04-17T06:09:24.674159+0000","last_became_peered":"2024-04-17T06:09:24.674159+0000","last_unstale":"2024-04-17T06:09:25.660166+0000","last_undegraded":"2024-04-17T06:09:25.660166+0000","last_fullsized":"2024-04-17T06:09:25.660166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:40.236016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010314+0000","last_change":"2024-04-17T06:09:20.638332+0000","last_active":"2024-04-17T06:09:26.010314+0000","last_peered":"2024-04-17T06:09:26.010314+0000","last_clean":"2024-04-17T06:09:26.010314+0000","last_became_active":"2024-04-17T06:09:20.638096+0000","last_became_peered":"2024-04-17T06:09:20.638096+0000","last_unstale":"2024-04-17T06:09:26.010314+0000","last_undegraded":"2024-04-17T06:09:26.010314+0000","last_fullsized":"2024-04-17T06:09:26.010314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012032+0000","last_change":"2024-04-17T06:09:19.627827+0000","last_active":"2024-04-17T06:09:26.012032+0000","last_peered":"2024-04-17T06:09:26.012032+0000","last_clean":"2024-04-17T06:09:26.012032+0000","last_became_active":"2024-04-17T06:09:19.627529+0000","last_became_peered":"2024-04-17T06:09:19.627529+0000","last_unstale":"2024-04-17T06:09:26.012032+0000","last_undegraded":"2024-04-17T06:09:26.012032+0000","last_fullsized":"2024-04-17T06:09:26.012032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659711+0000","last_change":"2024-04-17T06:09:19.623047+0000","last_active":"2024-04-17T06:09:25.659711+0000","last_peered":"2024-04-17T06:09:25.659711+0000","last_clean":"2024-04-17T06:09:25.659711+0000","last_became_active":"2024-04-17T06:09:19.622700+0000","last_became_peered":"2024-04-17T06:09:19.622700+0000","last_unstale":"2024-04-17T06:09:25.659711+0000","last_undegraded":"2024-04-17T06:09:25.659711+0000","last_fullsized":"2024-04-17T06:09:25.659711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011732+0000","last_change":"2024-04-17T06:09:20.630941+0000","last_active":"2024-04-17T06:09:26.011732+0000","last_peered":"2024-04-17T06:09:26.011732+0000","last_clean":"2024-04-17T06:09:26.011732+0000","last_became_active":"2024-04-17T06:09:20.630680+0000","last_became_peered":"2024-04-17T06:09:20.630680+0000","last_unstale":"2024-04-17T06:09:26.011732+0000","last_undegraded":"2024-04-17T06:09:26.011732+0000","last_fullsized":"2024-04-17T06:09:26.011732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661370+0000","last_change":"2024-04-17T06:09:24.668919+0000","last_active":"2024-04-17T06:09:25.661370+0000","last_peered":"2024-04-17T06:09:25.661370+0000","last_clean":"2024-04-17T06:09:25.661370+0000","last_became_active":"2024-04-17T06:09:24.668456+0000","last_became_peered":"2024-04-17T06:09:24.668456+0000","last_unstale":"2024-04-17T06:09:25.661370+0000","last_undegraded":"2024-04-17T06:09:25.661370+0000","last_fullsized":"2024-04-17T06:09:25.661370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:57.434392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011442+0000","last_change":"2024-04-17T06:09:19.622862+0000","last_active":"2024-04-17T06:09:26.011442+0000","last_peered":"2024-04-17T06:09:26.011442+0000","last_clean":"2024-04-17T06:09:26.011442+0000","last_became_active":"2024-04-17T06:09:19.622514+0000","last_became_peered":"2024-04-17T06:09:19.622514+0000","last_unstale":"2024-04-17T06:09:26.011442+0000","last_undegraded":"2024-04-17T06:09:26.011442+0000","last_fullsized":"2024-04-17T06:09:26.011442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011374+0000","last_change":"2024-04-17T06:09:20.638095+0000","last_active":"2024-04-17T06:09:26.011374+0000","last_peered":"2024-04-17T06:09:26.011374+0000","last_clean":"2024-04-17T06:09:26.011374+0000","last_became_active":"2024-04-17T06:09:20.637805+0000","last_became_peered":"2024-04-17T06:09:20.637805+0000","last_unstale":"2024-04-17T06:09:26.011374+0000","last_undegraded":"2024-04-17T06:09:26.011374+0000","last_fullsized":"2024-04-17T06:09:26.011374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:48.980595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012082+0000","last_change":"2024-04-17T06:09:24.675683+0000","last_active":"2024-04-17T06:09:26.012082+0000","last_peered":"2024-04-17T06:09:26.012082+0000","last_clean":"2024-04-17T06:09:26.012082+0000","last_became_active":"2024-04-17T06:09:24.675309+0000","last_became_peered":"2024-04-17T06:09:24.675309+0000","last_unstale":"2024-04-17T06:09:26.012082+0000","last_undegraded":"2024-04-17T06:09:26.012082+0000","last_fullsized":"2024-04-17T06:09:26.012082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:10.888244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659223+0000","last_change":"2024-04-17T06:09:20.640987+0000","last_active":"2024-04-17T06:09:25.659223+0000","last_peered":"2024-04-17T06:09:25.659223+0000","last_clean":"2024-04-17T06:09:25.659223+0000","last_became_active":"2024-04-17T06:09:20.640396+0000","last_became_peered":"2024-04-17T06:09:20.640396+0000","last_unstale":"2024-04-17T06:09:25.659223+0000","last_undegraded":"2024-04-17T06:09:25.659223+0000","last_fullsized":"2024-04-17T06:09:25.659223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660950+0000","last_change":"2024-04-17T06:09:20.642450+0000","last_active":"2024-04-17T06:09:25.660950+0000","last_peered":"2024-04-17T06:09:25.660950+0000","last_clean":"2024-04-17T06:09:25.660950+0000","last_became_active":"2024-04-17T06:09:20.639465+0000","last_became_peered":"2024-04-17T06:09:20.639465+0000","last_unstale":"2024-04-17T06:09:25.660950+0000","last_undegraded":"2024-04-17T06:09:25.660950+0000","last_fullsized":"2024-04-17T06:09:25.660950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:08.393904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660526+0000","last_change":"2024-04-17T06:09:20.652419+0000","last_active":"2024-04-17T06:09:25.660526+0000","last_peered":"2024-04-17T06:09:25.660526+0000","last_clean":"2024-04-17T06:09:25.660526+0000","last_became_active":"2024-04-17T06:09:20.652139+0000","last_became_peered":"2024-04-17T06:09:20.652139+0000","last_unstale":"2024-04-17T06:09:25.660526+0000","last_undegraded":"2024-04-17T06:09:25.660526+0000","last_fullsized":"2024-04-17T06:09:25.660526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011662+0000","last_change":"2024-04-17T06:09:20.636986+0000","last_active":"2024-04-17T06:09:26.011662+0000","last_peered":"2024-04-17T06:09:26.011662+0000","last_clean":"2024-04-17T06:09:26.011662+0000","last_became_active":"2024-04-17T06:09:20.636761+0000","last_became_peered":"2024-04-17T06:09:20.636761+0000","last_unstale":"2024-04-17T06:09:26.011662+0000","last_undegraded":"2024-04-17T06:09:26.011662+0000","last_fullsized":"2024-04-17T06:09:26.011662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012351+0000","last_change":"2024-04-17T06:09:20.646197+0000","last_active":"2024-04-17T06:09:26.012351+0000","last_peered":"2024-04-17T06:09:26.012351+0000","last_clean":"2024-04-17T06:09:26.012351+0000","last_became_active":"2024-04-17T06:09:20.645965+0000","last_became_peered":"2024-04-17T06:09:20.645965+0000","last_unstale":"2024-04-17T06:09:26.012351+0000","last_undegraded":"2024-04-17T06:09:26.012351+0000","last_fullsized":"2024-04-17T06:09:26.012351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011415+0000","last_change":"2024-04-17T06:09:20.644696+0000","last_active":"2024-04-17T06:09:26.011415+0000","last_peered":"2024-04-17T06:09:26.011415+0000","last_clean":"2024-04-17T06:09:26.011415+0000","last_became_active":"2024-04-17T06:09:20.644322+0000","last_became_peered":"2024-04-17T06:09:20.644322+0000","last_unstale":"2024-04-17T06:09:26.011415+0000","last_undegraded":"2024-04-17T06:09:26.011415+0000","last_fullsized":"2024-04-17T06:09:26.011415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660919+0000","last_change":"2024-04-17T06:09:20.650016+0000","last_active":"2024-04-17T06:09:25.660919+0000","last_peered":"2024-04-17T06:09:25.660919+0000","last_clean":"2024-04-17T06:09:25.660919+0000","last_became_active":"2024-04-17T06:09:20.649252+0000","last_became_peered":"2024-04-17T06:09:20.649252+0000","last_unstale":"2024-04-17T06:09:25.660919+0000","last_undegraded":"2024-04-17T06:09:25.660919+0000","last_fullsized":"2024-04-17T06:09:25.660919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010916+0000","last_change":"2024-04-17T06:09:20.646003+0000","last_active":"2024-04-17T06:09:26.010916+0000","last_peered":"2024-04-17T06:09:26.010916+0000","last_clean":"2024-04-17T06:09:26.010916+0000","last_became_active":"2024-04-17T06:09:20.645732+0000","last_became_peered":"2024-04-17T06:09:20.645732+0000","last_unstale":"2024-04-17T06:09:26.010916+0000","last_undegraded":"2024-04-17T06:09:26.010916+0000","last_fullsized":"2024-04-17T06:09:26.010916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659748+0000","last_change":"2024-04-17T06:09:20.650803+0000","last_active":"2024-04-17T06:09:25.659748+0000","last_peered":"2024-04-17T06:09:25.659748+0000","last_clean":"2024-04-17T06:09:25.659748+0000","last_became_active":"2024-04-17T06:09:20.649856+0000","last_became_peered":"2024-04-17T06:09:20.649856+0000","last_unstale":"2024-04-17T06:09:25.659748+0000","last_undegraded":"2024-04-17T06:09:25.659748+0000","last_fullsized":"2024-04-17T06:09:25.659748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011942+0000","last_change":"2024-04-17T06:09:20.643474+0000","last_active":"2024-04-17T06:09:26.011942+0000","last_peered":"2024-04-17T06:09:26.011942+0000","last_clean":"2024-04-17T06:09:26.011942+0000","last_became_active":"2024-04-17T06:09:20.643125+0000","last_became_peered":"2024-04-17T06:09:20.643125+0000","last_unstale":"2024-04-17T06:09:26.011942+0000","last_undegraded":"2024-04-17T06:09:26.011942+0000","last_fullsized":"2024-04-17T06:09:26.011942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660299+0000","last_change":"2024-04-17T06:09:20.646984+0000","last_active":"2024-04-17T06:09:25.660299+0000","last_peered":"2024-04-17T06:09:25.660299+0000","last_clean":"2024-04-17T06:09:25.660299+0000","last_became_active":"2024-04-17T06:09:20.646731+0000","last_became_peered":"2024-04-17T06:09:20.646731+0000","last_unstale":"2024-04-17T06:09:25.660299+0000","last_undegraded":"2024-04-17T06:09:25.660299+0000","last_fullsized":"2024-04-17T06:09:25.660299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:11.442995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011435+0000","last_change":"2024-04-17T06:09:20.648582+0000","last_active":"2024-04-17T06:09:26.011435+0000","last_peered":"2024-04-17T06:09:26.011435+0000","last_clean":"2024-04-17T06:09:26.011435+0000","last_became_active":"2024-04-17T06:09:20.648345+0000","last_became_peered":"2024-04-17T06:09:20.648345+0000","last_unstale":"2024-04-17T06:09:26.011435+0000","last_undegraded":"2024-04-17T06:09:26.011435+0000","last_fullsized":"2024-04-17T06:09:26.011435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660359+0000","last_change":"2024-04-17T06:09:20.647687+0000","last_active":"2024-04-17T06:09:25.660359+0000","last_peered":"2024-04-17T06:09:25.660359+0000","last_clean":"2024-04-17T06:09:25.660359+0000","last_became_active":"2024-04-17T06:09:20.646672+0000","last_became_peered":"2024-04-17T06:09:20.646672+0000","last_unstale":"2024-04-17T06:09:25.660359+0000","last_undegraded":"2024-04-17T06:09:25.660359+0000","last_fullsized":"2024-04-17T06:09:25.660359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011526+0000","last_change":"2024-04-17T06:09:20.646327+0000","last_active":"2024-04-17T06:09:26.011526+0000","last_peered":"2024-04-17T06:09:26.011526+0000","last_clean":"2024-04-17T06:09:26.011526+0000","last_became_active":"2024-04-17T06:09:20.645768+0000","last_became_peered":"2024-04-17T06:09:20.645768+0000","last_unstale":"2024-04-17T06:09:26.011526+0000","last_undegraded":"2024-04-17T06:09:26.011526+0000","last_fullsized":"2024-04-17T06:09:26.011526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:55.368933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661388+0000","last_change":"2024-04-17T06:09:20.638979+0000","last_active":"2024-04-17T06:09:25.661388+0000","last_peered":"2024-04-17T06:09:25.661388+0000","last_clean":"2024-04-17T06:09:25.661388+0000","last_became_active":"2024-04-17T06:09:20.638679+0000","last_became_peered":"2024-04-17T06:09:20.638679+0000","last_unstale":"2024-04-17T06:09:25.661388+0000","last_undegraded":"2024-04-17T06:09:25.661388+0000","last_fullsized":"2024-04-17T06:09:25.661388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:08.892063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661039+0000","last_change":"2024-04-17T06:09:20.647598+0000","last_active":"2024-04-17T06:09:25.661039+0000","last_peered":"2024-04-17T06:09:25.661039+0000","last_clean":"2024-04-17T06:09:25.661039+0000","last_became_active":"2024-04-17T06:09:20.647352+0000","last_became_peered":"2024-04-17T06:09:20.647352+0000","last_unstale":"2024-04-17T06:09:25.661039+0000","last_undegraded":"2024-04-17T06:09:25.661039+0000","last_fullsized":"2024-04-17T06:09:25.661039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:16.143995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659697+0000","last_change":"2024-04-17T06:09:20.634824+0000","last_active":"2024-04-17T06:09:25.659697+0000","last_peered":"2024-04-17T06:09:25.659697+0000","last_clean":"2024-04-17T06:09:25.659697+0000","last_became_active":"2024-04-17T06:09:20.634526+0000","last_became_peered":"2024-04-17T06:09:20.634526+0000","last_unstale":"2024-04-17T06:09:25.659697+0000","last_undegraded":"2024-04-17T06:09:25.659697+0000","last_fullsized":"2024-04-17T06:09:25.659697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010538+0000","last_change":"2024-04-17T06:09:20.643072+0000","last_active":"2024-04-17T06:09:26.010538+0000","last_peered":"2024-04-17T06:09:26.010538+0000","last_clean":"2024-04-17T06:09:26.010538+0000","last_became_active":"2024-04-17T06:09:20.642812+0000","last_became_peered":"2024-04-17T06:09:20.642812+0000","last_unstale":"2024-04-17T06:09:26.010538+0000","last_undegraded":"2024-04-17T06:09:26.010538+0000","last_fullsized":"2024-04-17T06:09:26.010538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010750+0000","last_change":"2024-04-17T06:09:20.636421+0000","last_active":"2024-04-17T06:09:26.010750+0000","last_peered":"2024-04-17T06:09:26.010750+0000","last_clean":"2024-04-17T06:09:26.010750+0000","last_became_active":"2024-04-17T06:09:20.636165+0000","last_became_peered":"2024-04-17T06:09:20.636165+0000","last_unstale":"2024-04-17T06:09:26.010750+0000","last_undegraded":"2024-04-17T06:09:26.010750+0000","last_fullsized":"2024-04-17T06:09:26.010750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:41.556474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011251+0000","last_change":"2024-04-17T06:09:20.639331+0000","last_active":"2024-04-17T06:09:26.011251+0000","last_peered":"2024-04-17T06:09:26.011251+0000","last_clean":"2024-04-17T06:09:26.011251+0000","last_became_active":"2024-04-17T06:09:20.639051+0000","last_became_peered":"2024-04-17T06:09:20.639051+0000","last_unstale":"2024-04-17T06:09:26.011251+0000","last_undegraded":"2024-04-17T06:09:26.011251+0000","last_fullsized":"2024-04-17T06:09:26.011251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010813+0000","last_change":"2024-04-17T06:09:20.639239+0000","last_active":"2024-04-17T06:09:26.010813+0000","last_peered":"2024-04-17T06:09:26.010813+0000","last_clean":"2024-04-17T06:09:26.010813+0000","last_became_active":"2024-04-17T06:09:20.638982+0000","last_became_peered":"2024-04-17T06:09:20.638982+0000","last_unstale":"2024-04-17T06:09:26.010813+0000","last_undegraded":"2024-04-17T06:09:26.010813+0000","last_fullsized":"2024-04-17T06:09:26.010813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:08.892063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659939+0000","last_change":"2024-04-17T06:09:20.639508+0000","last_active":"2024-04-17T06:09:25.659939+0000","last_peered":"2024-04-17T06:09:25.659939+0000","last_clean":"2024-04-17T06:09:25.659939+0000","last_became_active":"2024-04-17T06:09:20.639151+0000","last_became_peered":"2024-04-17T06:09:20.639151+0000","last_unstale":"2024-04-17T06:09:25.659939+0000","last_undegraded":"2024-04-17T06:09:25.659939+0000","last_fullsized":"2024-04-17T06:09:25.659939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660890+0000","last_change":"2024-04-17T06:09:20.639084+0000","last_active":"2024-04-17T06:09:25.660890+0000","last_peered":"2024-04-17T06:09:25.660890+0000","last_clean":"2024-04-17T06:09:25.660890+0000","last_became_active":"2024-04-17T06:09:20.638793+0000","last_became_peered":"2024-04-17T06:09:20.638793+0000","last_unstale":"2024-04-17T06:09:25.660890+0000","last_undegraded":"2024-04-17T06:09:25.660890+0000","last_fullsized":"2024-04-17T06:09:25.660890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011592+0000","last_change":"2024-04-17T06:09:20.645741+0000","last_active":"2024-04-17T06:09:26.011592+0000","last_peered":"2024-04-17T06:09:26.011592+0000","last_clean":"2024-04-17T06:09:26.011592+0000","last_became_active":"2024-04-17T06:09:20.645168+0000","last_became_peered":"2024-04-17T06:09:20.645168+0000","last_unstale":"2024-04-17T06:09:26.011592+0000","last_undegraded":"2024-04-17T06:09:26.011592+0000","last_fullsized":"2024-04-17T06:09:26.011592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:16.143995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010798+0000","last_change":"2024-04-17T06:09:20.656219+0000","last_active":"2024-04-17T06:09:26.010798+0000","last_peered":"2024-04-17T06:09:26.010798+0000","last_clean":"2024-04-17T06:09:26.010798+0000","last_became_active":"2024-04-17T06:09:20.655978+0000","last_became_peered":"2024-04-17T06:09:20.655978+0000","last_unstale":"2024-04-17T06:09:26.010798+0000","last_undegraded":"2024-04-17T06:09:26.010798+0000","last_fullsized":"2024-04-17T06:09:26.010798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010761+0000","last_change":"2024-04-17T06:09:20.650090+0000","last_active":"2024-04-17T06:09:26.010761+0000","last_peered":"2024-04-17T06:09:26.010761+0000","last_clean":"2024-04-17T06:09:26.010761+0000","last_became_active":"2024-04-17T06:09:20.649596+0000","last_became_peered":"2024-04-17T06:09:20.649596+0000","last_unstale":"2024-04-17T06:09:26.010761+0000","last_undegraded":"2024-04-17T06:09:26.010761+0000","last_fullsized":"2024-04-17T06:09:26.010761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660177+0000","last_change":"2024-04-17T06:09:20.634204+0000","last_active":"2024-04-17T06:09:25.660177+0000","last_peered":"2024-04-17T06:09:25.660177+0000","last_clean":"2024-04-17T06:09:25.660177+0000","last_became_active":"2024-04-17T06:09:20.633924+0000","last_became_peered":"2024-04-17T06:09:20.633924+0000","last_unstale":"2024-04-17T06:09:25.660177+0000","last_undegraded":"2024-04-17T06:09:25.660177+0000","last_fullsized":"2024-04-17T06:09:25.660177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010796+0000","last_change":"2024-04-17T06:09:20.645437+0000","last_active":"2024-04-17T06:09:26.010796+0000","last_peered":"2024-04-17T06:09:26.010796+0000","last_clean":"2024-04-17T06:09:26.010796+0000","last_became_active":"2024-04-17T06:09:20.645015+0000","last_became_peered":"2024-04-17T06:09:20.645015+0000","last_unstale":"2024-04-17T06:09:26.010796+0000","last_undegraded":"2024-04-17T06:09:26.010796+0000","last_fullsized":"2024-04-17T06:09:26.010796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660367+0000","last_change":"2024-04-17T06:09:20.650436+0000","last_active":"2024-04-17T06:09:25.660367+0000","last_peered":"2024-04-17T06:09:25.660367+0000","last_clean":"2024-04-17T06:09:25.660367+0000","last_became_active":"2024-04-17T06:09:20.650132+0000","last_became_peered":"2024-04-17T06:09:20.650132+0000","last_unstale":"2024-04-17T06:09:25.660367+0000","last_undegraded":"2024-04-17T06:09:25.660367+0000","last_fullsized":"2024-04-17T06:09:25.660367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011167+0000","last_change":"2024-04-17T06:09:20.634255+0000","last_active":"2024-04-17T06:09:26.011167+0000","last_peered":"2024-04-17T06:09:26.011167+0000","last_clean":"2024-04-17T06:09:26.011167+0000","last_became_active":"2024-04-17T06:09:20.633879+0000","last_became_peered":"2024-04-17T06:09:20.633879+0000","last_unstale":"2024-04-17T06:09:26.011167+0000","last_undegraded":"2024-04-17T06:09:26.011167+0000","last_fullsized":"2024-04-17T06:09:26.011167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:07.365672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010938+0000","last_change":"2024-04-17T06:09:20.635023+0000","last_active":"2024-04-17T06:09:26.010938+0000","last_peered":"2024-04-17T06:09:26.010938+0000","last_clean":"2024-04-17T06:09:26.010938+0000","last_became_active":"2024-04-17T06:09:20.634245+0000","last_became_peered":"2024-04-17T06:09:20.634245+0000","last_unstale":"2024-04-17T06:09:26.010938+0000","last_undegraded":"2024-04-17T06:09:26.010938+0000","last_fullsized":"2024-04-17T06:09:26.010938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011506+0000","last_change":"2024-04-17T06:09:20.653636+0000","last_active":"2024-04-17T06:09:26.011506+0000","last_peered":"2024-04-17T06:09:26.011506+0000","last_clean":"2024-04-17T06:09:26.011506+0000","last_became_active":"2024-04-17T06:09:20.653366+0000","last_became_peered":"2024-04-17T06:09:20.653366+0000","last_unstale":"2024-04-17T06:09:26.011506+0000","last_undegraded":"2024-04-17T06:09:26.011506+0000","last_fullsized":"2024-04-17T06:09:26.011506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:58.610924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T06:09:38.026 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T06:09:38.027 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-17T06:09:38.027 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-17T06:09:38.027 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-17T06:09:38.027 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-17T06:09:38.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.167+0000 7f9d78746640 1 Processor -- start 2024-04-17T06:09:38.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.167+0000 7f9d78746640 1 -- start start 2024-04-17T06:09:38.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d78746640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d700cef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d78746640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9d700cc470 0x7f9d700cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d78746640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9d700cea30 0x7f9d70072360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d78746640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9d700ad860 con 0x7f9d700cea30 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d78746640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9d7015eff0 con 0x7f9d700cc470 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d78746640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9d700cc970 con 0x7f9d7015ec10 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d76ce3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9d700cea30 0x7f9d70072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d76ce3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9d700cea30 0x7f9d70072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48594/0 (socket says 172.21.15.135:48594) 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d76ce3640 1 -- 172.21.15.135:0/1679302076 learned_addr learned my addr 172.21.15.135:0/1679302076 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d764e2640 1 --2- 172.21.15.135:0/1679302076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d700cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d75ce1640 1 --2- 172.21.15.135:0/1679302076 >> v2:172.21.15.190:3300/0 conn(0x7f9d700cc470 0x7f9d700cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d76ce3640 1 -- 172.21.15.135:0/1679302076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 msgr2=0x7f9d700cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d76ce3640 1 --2- 172.21.15.135:0/1679302076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d700cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d76ce3640 1 -- 172.21.15.135:0/1679302076 >> v2:172.21.15.190:3300/0 conn(0x7f9d700cc470 msgr2=0x7f9d700cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:38.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d76ce3640 1 --2- 172.21.15.135:0/1679302076 >> v2:172.21.15.190:3300/0 conn(0x7f9d700cc470 0x7f9d700cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d76ce3640 1 -- 172.21.15.135:0/1679302076 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d700ccc50 con 0x7f9d700cea30 2024-04-17T06:09:38.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.168+0000 7f9d764e2640 1 --2- 172.21.15.135:0/1679302076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d700cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:38.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.169+0000 7f9d76ce3640 1 --2- 172.21.15.135:0/1679302076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d70072360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d7006ccc0 tx=0x7f9d6c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d764ecdce93016 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.169+0000 7f9d754e0640 1 -- 172.21.15.135:0/1679302076 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c02e070 con 0x7f9d700cea30 2024-04-17T06:09:38.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.169+0000 7f9d754e0640 1 -- 172.21.15.135:0/1679302076 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d6c037070 con 0x7f9d700cea30 2024-04-17T06:09:38.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.169+0000 7f9d754e0640 1 -- 172.21.15.135:0/1679302076 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c031a00 con 0x7f9d700cea30 2024-04-17T06:09:38.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 -- 172.21.15.135:0/1679302076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 msgr2=0x7f9d70072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 --2- 172.21.15.135:0/1679302076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d70072360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9d7006ccc0 tx=0x7f9d6c02b880 comp rx=0 tx=0).stop 2024-04-17T06:09:38.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 -- 172.21.15.135:0/1679302076 shutdown_connections 2024-04-17T06:09:38.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 --2- 172.21.15.135:0/1679302076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d70072360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 --2- 172.21.15.135:0/1679302076 >> v2:172.21.15.190:3300/0 conn(0x7f9d700cc470 0x7f9d700cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 --2- 172.21.15.135:0/1679302076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d700cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 -- 172.21.15.135:0/1679302076 >> 172.21.15.135:0/1679302076 conn(0x7f9d700d0930 msgr2=0x7f9d700d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:38.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 -- 172.21.15.135:0/1679302076 shutdown_connections 2024-04-17T06:09:38.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.170+0000 7f9d78746640 1 -- 172.21.15.135:0/1679302076 wait complete. 2024-04-17T06:09:38.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.171+0000 7f9d78746640 1 Processor -- start 2024-04-17T06:09:38.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.171+0000 7f9d78746640 1 -- start start 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d78746640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d700cc470 0x7f9d701613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d78746640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d7006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d78746640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d70159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d78746640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9d700ce6c0 con 0x7f9d700cea30 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d78746640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9d700add90 con 0x7f9d700cc470 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d78746640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9d700ce840 con 0x7f9d7015ec10 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d7006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d7006fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48608/0 (socket says 172.21.15.135:48608) 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 -- 172.21.15.135:0/3527703502 learned_addr learned my addr 172.21.15.135:0/3527703502 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:38.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d76ce3640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d70159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d764e2640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d700cc470 0x7f9d701613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 -- 172.21.15.135:0/3527703502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 msgr2=0x7f9d70159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d70159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 -- 172.21.15.135:0/3527703502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d700cc470 msgr2=0x7f9d701613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d700cc470 0x7f9d701613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 -- 172.21.15.135:0/3527703502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7006e440 con 0x7f9d700cea30 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.172+0000 7f9d75ce1640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d7006fc10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e8a0 tx=0x7f9d6000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.173+0000 7f9d66ffd640 1 -- 172.21.15.135:0/3527703502 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60017070 con 0x7f9d700cea30 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.173+0000 7f9d66ffd640 1 -- 172.21.15.135:0/3527703502 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d6000ee90 con 0x7f9d700cea30 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.173+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7006e6b0 con 0x7f9d700cea30 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.173+0000 7f9d66ffd640 1 -- 172.21.15.135:0/3527703502 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000cb80 con 0x7f9d700cea30 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.173+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d70161930 con 0x7f9d700cea30 2024-04-17T06:09:38.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.173+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d700cef80 con 0x7f9d700cea30 2024-04-17T06:09:38.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.174+0000 7f9d66ffd640 1 -- 172.21.15.135:0/3527703502 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d6001a070 con 0x7f9d700cea30 2024-04-17T06:09:38.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.174+0000 7f9d66ffd640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9d5406c8f0 0x7f9d5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.174+0000 7f9d66ffd640 1 -- 172.21.15.135:0/3527703502 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9d6008bc60 con 0x7f9d700cea30 2024-04-17T06:09:38.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.174+0000 7f9d764e2640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9d5406c8f0 0x7f9d5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.175+0000 7f9d764e2640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9d5406c8f0 0x7f9d5406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d68003fd0 tx=0x7f9d68003ce0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.179+0000 7f9d66ffd640 1 -- 172.21.15.135:0/3527703502 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d700cef80 con 0x7f9d700cea30 2024-04-17T06:09:38.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.403+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9d70074940 con 0x7f9d700cea30 2024-04-17T06:09:38.406 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.406+0000 7f9d66ffd640 1 -- 172.21.15.135:0/3527703502 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174144 (secure 0 0 0) 0x7f9d60059de0 con 0x7f9d700cea30 2024-04-17T06:09:38.408 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:38.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9d5406c8f0 msgr2=0x7f9d5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9d5406c8f0 0x7f9d5406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d68003fd0 tx=0x7f9d68003ce0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 msgr2=0x7f9d7006fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d7006fc10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e8a0 tx=0x7f9d6000ed60 comp rx=0 tx=0).stop 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 shutdown_connections 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d7015ec10 0x7f9d70159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9d5406c8f0 0x7f9d5406edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d700cea30 0x7f9d7006fc10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 --2- 172.21.15.135:0/3527703502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d700cc470 0x7f9d701613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 >> 172.21.15.135:0/3527703502 conn(0x7f9d700d0930 msgr2=0x7f9d700c6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 shutdown_connections 2024-04-17T06:09:38.410 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.409+0000 7f9d78746640 1 -- 172.21.15.135:0/3527703502 wait complete. 2024-04-17T06:09:38.418 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":5,"flags":0,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":3190814219},{"type":"v1","addr":"172.21.15.135:6833","nonce":3190814219}]},"active_addr":"172.21.15.135:6833/3190814219","active_change":"2024-04-17T06:08:59.930001+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":2177462233}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":3667451176}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":1012689640}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.135:0","nonce":2719933990}]}]} 2024-04-17T06:09:38.421 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-17T06:09:38.421 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-17T06:09:38.421 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:38.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.526+0000 7f1dcbad3640 1 Processor -- start 2024-04-17T06:09:38.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.526+0000 7f1dcbad3640 1 -- start start 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dcbad3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc415ec90 0x7f1dc40ba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dcbad3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1dc40b36f0 0x7f1dc40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dcbad3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc40ba770 con 0x7f1dc40b6770 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dcbad3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc40d2b80 con 0x7f1dc40b36f0 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dcbad3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc40d3700 con 0x7f1dc415ec90 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dca070640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f1dc40b6770 0x7f1dc40d3b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37688/0 (socket says 172.21.15.135:37688) 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dca070640 1 -- 172.21.15.135:0/2355218007 learned_addr learned my addr 172.21.15.135:0/2355218007 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dc986f640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc415ec90 0x7f1dc40ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dc906e640 1 --2- 172.21.15.135:0/2355218007 >> v2:172.21.15.190:3300/0 conn(0x7f1dc40b36f0 0x7f1dc40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.529 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.527+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1247718057 0 0) 0x7f1dc40ba770 con 0x7f1dc40b6770 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1db0003520 con 0x7f1dc40b6770 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1817901058 0 0) 0x7f1db0003520 con 0x7f1dc40b6770 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc415ec90 msgr2=0x7f1dc40ba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc415ec90 0x7f1dc40ba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 >> v2:172.21.15.190:3300/0 conn(0x7f1dc40b36f0 msgr2=0x7f1dc40b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 --2- 172.21.15.135:0/2355218007 >> v2:172.21.15.190:3300/0 conn(0x7f1dc40b36f0 0x7f1dc40b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc415f070 con 0x7f1dc40b6770 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f1dbc0019f0 con 0x7f1dc40b6770 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 >> v1:172.21.15.135:6789/0 conn(0x7f1dc40b6770 legacy=0x7f1dc40d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db00039b0 0x7f1db0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1db0004800 0x7f1db0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc415ec90 0x7f1db00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc40d2b80 con 0x7f1db0004800 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc40d3700 con 0x7f1db00039b0 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1db000b9c0 con 0x7f1dc415ec90 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc906e640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1db0004800 0x7f1db0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc986f640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db00039b0 0x7f1db0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dca070640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc415ec90 0x7f1db00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc906e640 1 -- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc415ec90 msgr2=0x7f1db00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc906e640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc415ec90 0x7f1db00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc906e640 1 -- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db00039b0 msgr2=0x7f1db0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc906e640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db00039b0 0x7f1db0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc906e640 1 -- 172.21.15.135:0/2355218007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db0003520 con 0x7f1db0004800 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.528+0000 7f1dc986f640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db00039b0 0x7f1db0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:38.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.529+0000 7f1dc906e640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1db0004800 0x7f1db0004c80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ac0 tx=0x7f1db0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3382ec8ee599a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.529+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4039070 con 0x7f1db0004800 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.529+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db402c710 con 0x7f1db0004800 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.529+0000 7f1dc886d640 1 -- 172.21.15.135:0/2355218007 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4009690 con 0x7f1db0004800 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.530+0000 7f1dcbad3640 1 -- 172.21.15.135:0/2355218007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1db0004800 msgr2=0x7f1db0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.530+0000 7f1dcbad3640 1 --2- 172.21.15.135:0/2355218007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1db0004800 0x7f1db0004c80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ac0 tx=0x7f1db0005830 comp rx=0 tx=0).stop 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.530+0000 7f1dca070640 1 -- 172.21.15.135:0/2355218007 reap_dead start 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.530+0000 7f1dcbad3640 1 -- 172.21.15.135:0/2355218007 shutdown_connections 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.530+0000 7f1dcbad3640 1 -- 172.21.15.135:0/2355218007 >> 172.21.15.135:0/2355218007 conn(0x7f1dc40d17b0 msgr2=0x7f1dc40d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.530+0000 7f1dcbad3640 1 -- 172.21.15.135:0/2355218007 shutdown_connections 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.530+0000 7f1dcbad3640 1 -- 172.21.15.135:0/2355218007 wait complete. 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.531+0000 7f1dcbad3640 1 Processor -- start 2024-04-17T06:09:38.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.531+0000 7f1dcbad3640 1 -- start start 2024-04-17T06:09:38.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dcbad3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db0004800 0x7f1dc40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dcbad3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dc4161c10 0x7f1dc415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dcbad3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40ca9f0 0x7f1dc40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dcbad3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc40d3390 con 0x7f1dc4161c10 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dcbad3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc40d30b0 con 0x7f1db0004800 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dcbad3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc40b7f80 con 0x7f1dc40ca9f0 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dc4161c10 0x7f1dc415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dc4161c10 0x7f1dc415b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48624/0 (socket says 172.21.15.135:48624) 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 -- 172.21.15.135:0/1089994805 learned_addr learned my addr 172.21.15.135:0/1089994805 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dca070640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40ca9f0 0x7f1dc40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc986f640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db0004800 0x7f1dc40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 -- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40ca9f0 msgr2=0x7f1dc40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40ca9f0 0x7f1dc40cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 -- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db0004800 msgr2=0x7f1dc40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db0004800 0x7f1dc40cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 -- 172.21.15.135:0/1089994805 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc415b5e0 con 0x7f1dc4161c10 2024-04-17T06:09:38.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc986f640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db0004800 0x7f1dc40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dc906e640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dc4161c10 0x7f1dc415b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1db4037040 tx=0x7f1db4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dba7fc640 1 -- 172.21.15.135:0/1089994805 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4042020 con 0x7f1dc4161c10 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dba7fc640 1 -- 172.21.15.135:0/1089994805 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db403a040 con 0x7f1dc4161c10 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dba7fc640 1 -- 172.21.15.135:0/1089994805 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4039070 con 0x7f1dc4161c10 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.532+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc40cbfc0 con 0x7f1dc4161c10 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.533+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc40cc580 con 0x7f1dc4161c10 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.534+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc4074940 con 0x7f1dc4161c10 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.534+0000 7f1dba7fc640 1 -- 172.21.15.135:0/1089994805 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db4045020 con 0x7f1dc4161c10 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.535+0000 7f1dba7fc640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1da806c920 0x7f1da806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.535+0000 7f1dba7fc640 1 -- 172.21.15.135:0/1089994805 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1db40ad5f0 con 0x7f1dc4161c10 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.535+0000 7f1dc986f640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1da806c920 0x7f1da806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.535+0000 7f1dc986f640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1da806c920 0x7f1da806ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009c00 tx=0x7f1dc0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.540+0000 7f1dba7fc640 1 -- 172.21.15.135:0/1089994805 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1db404c030 con 0x7f1dc4161c10 2024-04-17T06:09:38.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.722+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dc406dd80 con 0x7f1dc4161c10 2024-04-17T06:09:38.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.723+0000 7f1dba7fc640 1 -- 172.21.15.135:0/1089994805 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f1db407b6e0 con 0x7f1dc4161c10 2024-04-17T06:09:38.721 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:38.722 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1da806c920 msgr2=0x7f1da806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1da806c920 0x7f1da806ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009c00 tx=0x7f1dc0008040 comp rx=0 tx=0).stop 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dc4161c10 msgr2=0x7f1dc415b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dc4161c10 0x7f1dc415b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1db4037040 tx=0x7f1db4004730 comp rx=0 tx=0).stop 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 shutdown_connections 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1dc40ca9f0 0x7f1dc40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1da806c920 0x7f1da806ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1dc4161c10 0x7f1dc415b0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 --2- 172.21.15.135:0/1089994805 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1db0004800 0x7f1dc40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 >> 172.21.15.135:0/1089994805 conn(0x7f1dc40d17b0 msgr2=0x7f1dc40cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:38.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.725+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 shutdown_connections 2024-04-17T06:09:38.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.726+0000 7f1dcbad3640 1 -- 172.21.15.135:0/1089994805 wait complete. 2024-04-17T06:09:38.735 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-17T06:09:38.735 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:38.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.840+0000 7f9488cba640 1 Processor -- start 2024-04-17T06:09:38.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.840+0000 7f9488cba640 1 -- start start 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f9488cba640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f948415cc60 0x7f94840d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f9488cba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 0x7f948409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f9488cba640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f948415d040 con 0x7f948409af30 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f9488cba640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f94840d13c0 con 0x7f948415cc60 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f9488cba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f94840b1d20 con 0x7f94840d0490 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f948259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f948415cc60 0x7f94840d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f948259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f948415cc60 0x7f94840d0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48636/0 (socket says 172.21.15.135:48636) 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f948259c640 1 -- 172.21.15.135:0/1492460346 learned_addr learned my addr 172.21.15.135:0/1492460346 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:38.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f9479d9b640 1 --2- 172.21.15.135:0/1492460346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 0x7f948409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f948259c640 1 -- 172.21.15.135:0/1492460346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 msgr2=0x7f948409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:38.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f948259c640 1 --2- 172.21.15.135:0/1492460346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 0x7f948409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f948259c640 1 -- 172.21.15.135:0/1492460346 >> v1:172.21.15.190:6789/0 conn(0x7f948409af30 legacy=0x7f94840cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:38.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.841+0000 7f948259c640 1 -- 172.21.15.135:0/1492460346 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94840b2000 con 0x7f948415cc60 2024-04-17T06:09:38.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.842+0000 7f948259c640 1 --2- 172.21.15.135:0/1492460346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948415cc60 0x7f94840d0b90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9474009870 tx=0x7f947402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365a88a40a182675 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.842+0000 7f9481d9b640 1 -- 172.21.15.135:0/1492460346 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474034070 con 0x7f948415cc60 2024-04-17T06:09:38.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.842+0000 7f9481d9b640 1 -- 172.21.15.135:0/1492460346 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9474030840 con 0x7f948415cc60 2024-04-17T06:09:38.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.842+0000 7f9481d9b640 1 -- 172.21.15.135:0/1492460346 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474030c10 con 0x7f948415cc60 2024-04-17T06:09:38.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.843+0000 7f9488cba640 1 -- 172.21.15.135:0/1492460346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948415cc60 msgr2=0x7f94840d0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.843+0000 7f9488cba640 1 --2- 172.21.15.135:0/1492460346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948415cc60 0x7f94840d0b90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9474009870 tx=0x7f947402b980 comp rx=0 tx=0).stop 2024-04-17T06:09:38.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.843+0000 7f9488cba640 1 -- 172.21.15.135:0/1492460346 shutdown_connections 2024-04-17T06:09:38.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.843+0000 7f9488cba640 1 --2- 172.21.15.135:0/1492460346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 0x7f948409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.843+0000 7f9488cba640 1 --2- 172.21.15.135:0/1492460346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948415cc60 0x7f94840d0b90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.843+0000 7f9488cba640 1 -- 172.21.15.135:0/1492460346 >> 172.21.15.135:0/1492460346 conn(0x7f94840b41a0 msgr2=0x7f94840b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:38.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.843+0000 7f9488cba640 1 -- 172.21.15.135:0/1492460346 shutdown_connections 2024-04-17T06:09:38.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.843+0000 7f9488cba640 1 -- 172.21.15.135:0/1492460346 wait complete. 2024-04-17T06:09:38.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.844+0000 7f9488cba640 1 Processor -- start 2024-04-17T06:09:38.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.844+0000 7f9488cba640 1 -- start start 2024-04-17T06:09:38.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f9488cba640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948409af30 0x7f948406dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f9488cba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 0x7f94840c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f9488cba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f948415cc60 0x7f94840c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f9488cba640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94840d16a0 con 0x7f948409af30 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f9488cba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f94840d10d0 con 0x7f948415cc60 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f9488cba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f94840d1820 con 0x7f94840d0490 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f948259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948409af30 0x7f948406dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f948259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948409af30 0x7f948406dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48652/0 (socket says 172.21.15.135:48652) 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f948259c640 1 -- 172.21.15.135:0/2820512576 learned_addr learned my addr 172.21.15.135:0/2820512576 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f9479d9b640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 0x7f94840c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f9482d9d640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f948415cc60 0x7f94840c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f948259c640 1 -- 172.21.15.135:0/2820512576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 msgr2=0x7f94840c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f948259c640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 0x7f94840c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f948259c640 1 -- 172.21.15.135:0/2820512576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f948415cc60 msgr2=0x7f94840c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:38.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f948259c640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f948415cc60 0x7f94840c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.845+0000 7f948259c640 1 -- 172.21.15.135:0/2820512576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948406cc20 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.846+0000 7f948259c640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948409af30 0x7f948406dcc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9474000c00 tx=0x7f9474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.846+0000 7f947b7fe640 1 -- 172.21.15.135:0/2820512576 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474048070 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.846+0000 7f947b7fe640 1 -- 172.21.15.135:0/2820512576 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9474030640 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.846+0000 7f947b7fe640 1 -- 172.21.15.135:0/2820512576 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474034040 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.846+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948406cf00 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.846+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948406d4c0 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.847+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9484074940 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.847+0000 7f947b7fe640 1 -- 172.21.15.135:0/2820512576 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9474037070 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.848+0000 7f947b7fe640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f946406c8f0 0x7f946406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.848+0000 7f947b7fe640 1 -- 172.21.15.135:0/2820512576 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f94740adb30 con 0x7f948409af30 2024-04-17T06:09:38.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.848+0000 7f9479d9b640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f946406c8f0 0x7f946406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:38.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.848+0000 7f9479d9b640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f946406c8f0 0x7f946406edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94840cc880 tx=0x7f946c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:38.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:38.852+0000 7f947b7fe640 1 -- 172.21.15.135:0/2820512576 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f947407b780 con 0x7f948409af30 2024-04-17T06:09:39.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.029+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9484068d40 con 0x7f948409af30 2024-04-17T06:09:39.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.030+0000 7f947b7fe640 1 -- 172.21.15.135:0/2820512576 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f947407fa80 con 0x7f948409af30 2024-04-17T06:09:39.028 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:39.028 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f946406c8f0 msgr2=0x7f946406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f946406c8f0 0x7f946406edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94840cc880 tx=0x7f946c008040 comp rx=0 tx=0).stop 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948409af30 msgr2=0x7f948406dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948409af30 0x7f948406dcc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9474000c00 tx=0x7f9474004730 comp rx=0 tx=0).stop 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 shutdown_connections 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f948415cc60 0x7f94840c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94840d0490 0x7f94840c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f946406c8f0 0x7f946406edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 --2- 172.21.15.135:0/2820512576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f948409af30 0x7f948406dcc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 >> 172.21.15.135:0/2820512576 conn(0x7f94840b41a0 msgr2=0x7f94840718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 shutdown_connections 2024-04-17T06:09:39.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.032+0000 7f9488cba640 1 -- 172.21.15.135:0/2820512576 wait complete. 2024-04-17T06:09:39.042 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-17T06:09:39.043 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-17T06:09:39.043 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-17T06:09:39.043 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-17T06:09:39.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-17T06:09:39.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-17T06:09:39.044 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-17T06:09:39.045 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-17T06:09:39.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.223+0000 7faf44c35640 1 Processor -- start 2024-04-17T06:09:39.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.223+0000 7faf44c35640 1 -- start start 2024-04-17T06:09:39.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.224+0000 7faf44c35640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 0x7faf38038740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.224+0000 7faf44c35640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faf38038c80 0x7faf38039ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.224+0000 7faf44c35640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf380545e0 con 0x7faf380e1200 2024-04-17T06:09:39.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.224+0000 7faf44c35640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7faf38036580 con 0x7faf38038c80 2024-04-17T06:09:39.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.224+0000 7faf44c35640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faf38039310 con 0x7faf38036120 2024-04-17T06:09:39.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.224+0000 7faf3e59c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7faf380e1200 0x7faf38051d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37696/0 (socket says 172.21.15.135:37696) 2024-04-17T06:09:39.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7fee15cd9640 1 Processor -- start 2024-04-17T06:09:39.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7faf3e59c640 1 -- 172.21.15.135:0/431215519 learned_addr learned my addr 172.21.15.135:0/431215519 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:39.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7faf3dd9b640 1 --2- 172.21.15.135:0/431215519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 0x7faf38038740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7faf3ed9d640 1 --2- 172.21.15.135:0/431215519 >> v2:172.21.15.190:3300/0 conn(0x7faf38038c80 0x7faf38039ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7fee15cd9640 1 -- start start 2024-04-17T06:09:39.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7faf3ed9d640 1 -- 172.21.15.135:0/431215519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 msgr2=0x7faf38038740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7faf3ed9d640 1 --2- 172.21.15.135:0/431215519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 0x7faf38038740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7faf3ed9d640 1 -- 172.21.15.135:0/431215519 >> v1:172.21.15.135:6789/0 conn(0x7faf380e1200 legacy=0x7faf38051d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.225+0000 7faf3ed9d640 1 -- 172.21.15.135:0/431215519 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf380395f0 con 0x7faf38038c80 2024-04-17T06:09:39.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.226+0000 7fee15cd9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee080eacf0 0x7fee0804fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.226+0000 7fee15cd9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee08006e90 0x7fee08007320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.226+0000 7fee15cd9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee08053710 con 0x7fee0804ffc0 2024-04-17T06:09:39.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.226+0000 7fee15cd9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fee08054320 con 0x7fee08006e90 2024-04-17T06:09:39.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.226+0000 7fee15cd9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fee080539f0 con 0x7fee080eacf0 2024-04-17T06:09:39.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fee0f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee080eacf0 0x7fee0804fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fee0f7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee080eacf0 0x7fee0804fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:49804/0 (socket says 172.21.15.135:49804) 2024-04-17T06:09:39.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fee0f7fe640 1 -- 172.21.15.135:0/484892291 learned_addr learned my addr 172.21.15.135:0/484892291 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7faf3ed9d640 1 --2- 172.21.15.135:0/431215519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf38038c80 0x7faf38039ef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf40000b60 tx=0x7faf400a7460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6620a7fb3b67b08e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7f9f3279d640 1 Processor -- start 2024-04-17T06:09:39.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fee0f7fe640 1 -- 172.21.15.135:0/484892291 >> v2:172.21.15.190:3300/0 conn(0x7fee08006e90 msgr2=0x7fee08007320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T06:09:39.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fee0f7fe640 1 --2- 172.21.15.135:0/484892291 >> v2:172.21.15.190:3300/0 conn(0x7fee08006e90 0x7fee08007320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fee0f7fe640 1 -- 172.21.15.135:0/484892291 >> v1:172.21.15.135:6789/0 conn(0x7fee0804ffc0 legacy=0x7fee08037bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fee0f7fe640 1 -- 172.21.15.135:0/484892291 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee08053cd0 con 0x7fee080eacf0 2024-04-17T06:09:39.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7faf3d59a640 1 -- 172.21.15.135:0/431215519 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400b3070 con 0x7faf38038c80 2024-04-17T06:09:39.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7faf3d59a640 1 -- 172.21.15.135:0/431215519 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf400ac7d0 con 0x7faf38038c80 2024-04-17T06:09:39.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.228+0000 7fce43667640 1 Processor -- start 2024-04-17T06:09:39.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fce43667640 1 -- start start 2024-04-17T06:09:39.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fe7952e4640 1 Processor -- start 2024-04-17T06:09:39.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7fe7952e4640 1 -- start start 2024-04-17T06:09:39.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fe7952e4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe7880e02c0 0x7fe7880e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fe7952e4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7880e40f0 0x7fe7880539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fe7952e4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe788056580 con 0x7fe7880dfee0 2024-04-17T06:09:39.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fe7952e4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe7880094a0 con 0x7fe7880e02c0 2024-04-17T06:09:39.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fe7952e4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe7880e4520 con 0x7fe7880e40f0 2024-04-17T06:09:39.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.227+0000 7f9f3279d640 1 -- start start 2024-04-17T06:09:39.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f3279d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f2c0afbe0 0x7f9f2c163410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f3279d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b37c0 0x7f9f2c1670c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f2bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f2c0afbe0 0x7f9f2c163410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f2b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b37c0 0x7f9f2c1670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f2bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9f2c0afbe0 0x7f9f2c163410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48656/0 (socket says 172.21.15.135:48656) 2024-04-17T06:09:39.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f2bfff640 1 -- 172.21.15.135:0/2610559835 learned_addr learned my addr 172.21.15.135:0/2610559835 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f3279d640 1 -- 172.21.15.135:0/2610559835 --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2c0b3c50 con 0x7f9f2c167600 2024-04-17T06:09:39.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f3279d640 1 -- 172.21.15.135:0/2610559835 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2c166c30 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f3279d640 1 -- 172.21.15.135:0/2610559835 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2c0affc0 con 0x7f9f2c0b37c0 2024-04-17T06:09:39.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7faf3d59a640 1 -- 172.21.15.135:0/431215519 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400acc40 con 0x7faf38038c80 2024-04-17T06:09:39.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fe78effd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe7880dfee0 0x7fe7880085c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39316/0 (socket says 172.21.15.135:39316) 2024-04-17T06:09:39.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fe78effd640 1 -- 172.21.15.135:0/3580087207 learned_addr learned my addr 172.21.15.135:0/3580087207 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:39.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fe78f7fe640 1 --2- 172.21.15.135:0/3580087207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7880e40f0 0x7fe7880539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7f9f2affd640 1 -- 172.21.15.135:0/2610559835 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257864653 0 0) 0x7f9f2c0b3c50 con 0x7f9f2c167600 2024-04-17T06:09:39.232 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fe78e7fc640 1 --2- 172.21.15.135:0/3580087207 >> v2:172.21.15.190:3300/0 conn(0x7fe7880e02c0 0x7fe7880e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7f9f2affd640 1 -- 172.21.15.135:0/2610559835 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f14003520 con 0x7f9f2c167600 2024-04-17T06:09:39.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fe78dffb640 1 -- 172.21.15.135:0/3580087207 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549832794 0 0) 0x7fe788056580 con 0x7fe7880dfee0 2024-04-17T06:09:39.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fe78dffb640 1 -- 172.21.15.135:0/3580087207 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe778003520 con 0x7fe7880dfee0 2024-04-17T06:09:39.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7f9f2bfff640 1 -- 172.21.15.135:0/2610559835 >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b37c0 msgr2=0x7f9f2c1670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7f9f2bfff640 1 --2- 172.21.15.135:0/2610559835 >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b37c0 0x7f9f2c1670c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7f9f2bfff640 1 -- 172.21.15.135:0/2610559835 >> v1:172.21.15.135:6789/0 conn(0x7f9f2c167600 legacy=0x7f9f2c06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7f9f2bfff640 1 -- 172.21.15.135:0/2610559835 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c163af0 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7f9f2b7fe640 1 --2- 172.21.15.135:0/2610559835 >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b37c0 0x7f9f2c1670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:39.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fe78f7fe640 1 -- 172.21.15.135:0/3580087207 >> v2:172.21.15.190:3300/0 conn(0x7fe7880e02c0 msgr2=0x7fe7880e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fe78f7fe640 1 --2- 172.21.15.135:0/3580087207 >> v2:172.21.15.190:3300/0 conn(0x7fe7880e02c0 0x7fe7880e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fe78f7fe640 1 -- 172.21.15.135:0/3580087207 >> v1:172.21.15.190:6789/0 conn(0x7fe7880dfee0 legacy=0x7fe7880085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fe78f7fe640 1 -- 172.21.15.135:0/3580087207 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe788056860 con 0x7fe7880e40f0 2024-04-17T06:09:39.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fce43667640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fce3c0af750 0x7fce3c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fce43667640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fce43667640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce3c0d0220 con 0x7fce3c06bed0 2024-04-17T06:09:39.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fce43667640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fce3c0b5f00 con 0x7fce3c0af750 2024-04-17T06:09:39.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.230+0000 7fce43667640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c0afb30 con 0x7fce3c0afd00 2024-04-17T06:09:39.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7faf44c35640 1 -- 172.21.15.135:0/431215519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf38038c80 msgr2=0x7faf38039ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7faf44c35640 1 --2- 172.21.15.135:0/431215519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf38038c80 0x7faf38039ef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf40000b60 tx=0x7faf400a7460 comp rx=0 tx=0).stop 2024-04-17T06:09:39.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fce41403640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fce3c0af750 0x7fce3c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.229+0000 7fee0f7fe640 1 --2- 172.21.15.135:0/484892291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee080eacf0 0x7fee0804fa80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee1002db70 tx=0x7fee100a7500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2271b02d22049aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fce41403640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fce3c0af750 0x7fce3c0cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48664/0 (socket says 172.21.15.135:48664) 2024-04-17T06:09:39.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fce41403640 1 -- 172.21.15.135:0/2654090830 learned_addr learned my addr 172.21.15.135:0/2654090830 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fee0e7fc640 1 -- 172.21.15.135:0/484892291 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100b3070 con 0x7fee080eacf0 2024-04-17T06:09:39.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fee0e7fc640 1 -- 172.21.15.135:0/484892291 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee100ac7d0 con 0x7fee080eacf0 2024-04-17T06:09:39.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fee0e7fc640 1 -- 172.21.15.135:0/484892291 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100acc40 con 0x7fee080eacf0 2024-04-17T06:09:39.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7faf44c35640 1 -- 172.21.15.135:0/431215519 shutdown_connections 2024-04-17T06:09:39.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7faf44c35640 1 --2- 172.21.15.135:0/431215519 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf38038c80 0x7faf38039ef0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7faf44c35640 1 --2- 172.21.15.135:0/431215519 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 0x7faf38038740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7faf44c35640 1 -- 172.21.15.135:0/431215519 >> 172.21.15.135:0/431215519 conn(0x7faf38053740 msgr2=0x7faf38053b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7f9f2bfff640 1 --2- 172.21.15.135:0/2610559835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 0x7f9f2c163410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0cd160 tx=0x7f9f1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32562c8825b32e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fce40c02640 1 --2- 172.21.15.135:0/2654090830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fce41403640 1 -- 172.21.15.135:0/2654090830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 msgr2=0x7fce3c06b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fe78f7fe640 1 --2- 172.21.15.135:0/3580087207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7880e40f0 0x7fe7880539e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe790000b60 tx=0x7fe7900a74c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc993ec22f8e333 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.231+0000 7fce41403640 1 --2- 172.21.15.135:0/2654090830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7fce41403640 1 -- 172.21.15.135:0/2654090830 >> v1:172.21.15.135:6789/0 conn(0x7fce3c06bed0 legacy=0x7fce3c0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7fce41403640 1 -- 172.21.15.135:0/2654090830 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c0b5980 con 0x7fce3c0af750 2024-04-17T06:09:39.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7fe78dffb640 1 -- 172.21.15.135:0/3580087207 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900b3070 con 0x7fe7880e40f0 2024-04-17T06:09:39.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7fe78dffb640 1 -- 172.21.15.135:0/3580087207 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7900ac7d0 con 0x7fe7880e40f0 2024-04-17T06:09:39.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7fe78dffb640 1 -- 172.21.15.135:0/3580087207 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900acc40 con 0x7fe7880e40f0 2024-04-17T06:09:39.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7f9f2affd640 1 -- 172.21.15.135:0/2610559835 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c035020 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7f9f2affd640 1 -- 172.21.15.135:0/2610559835 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1c031870 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7f9f2affd640 1 -- 172.21.15.135:0/2610559835 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c031c40 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7faf44c35640 1 -- 172.21.15.135:0/431215519 shutdown_connections 2024-04-17T06:09:39.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.232+0000 7faf44c35640 1 -- 172.21.15.135:0/431215519 wait complete. 2024-04-17T06:09:39.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7f9f3279d640 1 -- 172.21.15.135:0/2610559835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 msgr2=0x7f9f2c163410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fce41403640 1 --2- 172.21.15.135:0/2654090830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce3c0af750 0x7fce3c0cfce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce38009870 tx=0x7fce3802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03cd196f0d85126 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7f9f3279d640 1 --2- 172.21.15.135:0/2610559835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 0x7f9f2c163410 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0cd160 tx=0x7f9f1c02b910 comp rx=0 tx=0).stop 2024-04-17T06:09:39.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fe7952e4640 1 -- 172.21.15.135:0/3580087207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7880e40f0 msgr2=0x7fe7880539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fe7952e4640 1 --2- 172.21.15.135:0/3580087207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7880e40f0 0x7fe7880539e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe790000b60 tx=0x7fe7900a74c0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7f9f3279d640 1 -- 172.21.15.135:0/2610559835 shutdown_connections 2024-04-17T06:09:39.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fe7952e4640 1 -- 172.21.15.135:0/3580087207 shutdown_connections 2024-04-17T06:09:39.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fe7952e4640 1 --2- 172.21.15.135:0/3580087207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7880e40f0 0x7fe7880539e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fe7952e4640 1 --2- 172.21.15.135:0/3580087207 >> v2:172.21.15.190:3300/0 conn(0x7fe7880e02c0 0x7fe7880e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fe7952e4640 1 -- 172.21.15.135:0/3580087207 >> 172.21.15.135:0/3580087207 conn(0x7fe78801ef30 msgr2=0x7fe788055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fee15cd9640 1 -- 172.21.15.135:0/484892291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee080eacf0 msgr2=0x7fee0804fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fee15cd9640 1 --2- 172.21.15.135:0/484892291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee080eacf0 0x7fee0804fa80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee1002db70 tx=0x7fee100a7500 comp rx=0 tx=0).stop 2024-04-17T06:09:39.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fce33fff640 1 -- 172.21.15.135:0/2654090830 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38034070 con 0x7fce3c0af750 2024-04-17T06:09:39.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7f9f3279d640 1 --2- 172.21.15.135:0/2610559835 >> v2:172.21.15.190:3300/0 conn(0x7f9f2c0b37c0 0x7f9f2c1670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7f9f3279d640 1 --2- 172.21.15.135:0/2610559835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 0x7f9f2c163410 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7f9f3279d640 1 -- 172.21.15.135:0/2610559835 >> 172.21.15.135:0/2610559835 conn(0x7f9f2c0b2290 msgr2=0x7f9f2c0b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fce33fff640 1 -- 172.21.15.135:0/2654090830 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce38030840 con 0x7fce3c0af750 2024-04-17T06:09:39.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fce33fff640 1 -- 172.21.15.135:0/2654090830 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38030c10 con 0x7fce3c0af750 2024-04-17T06:09:39.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fe7952e4640 1 -- 172.21.15.135:0/3580087207 shutdown_connections 2024-04-17T06:09:39.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fee15cd9640 1 -- 172.21.15.135:0/484892291 shutdown_connections 2024-04-17T06:09:39.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fee15cd9640 1 --2- 172.21.15.135:0/484892291 >> v2:172.21.15.190:3300/0 conn(0x7fee08006e90 0x7fee08007320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fee15cd9640 1 --2- 172.21.15.135:0/484892291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee080eacf0 0x7fee0804fa80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fee15cd9640 1 -- 172.21.15.135:0/484892291 >> 172.21.15.135:0/484892291 conn(0x7fee08009fa0 msgr2=0x7fee0800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7fe7952e4640 1 -- 172.21.15.135:0/3580087207 wait complete. 2024-04-17T06:09:39.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.233+0000 7f9f3279d640 1 -- 172.21.15.135:0/2610559835 shutdown_connections 2024-04-17T06:09:39.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7fee15cd9640 1 -- 172.21.15.135:0/484892291 shutdown_connections 2024-04-17T06:09:39.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7f9f3279d640 1 -- 172.21.15.135:0/2610559835 wait complete. 2024-04-17T06:09:39.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7fce43667640 1 -- 172.21.15.135:0/2654090830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce3c0af750 msgr2=0x7fce3c0cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7fce43667640 1 --2- 172.21.15.135:0/2654090830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce3c0af750 0x7fce3c0cfce0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce38009870 tx=0x7fce3802b980 comp rx=0 tx=0).stop 2024-04-17T06:09:39.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7fce43667640 1 -- 172.21.15.135:0/2654090830 shutdown_connections 2024-04-17T06:09:39.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7fce43667640 1 --2- 172.21.15.135:0/2654090830 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7fce43667640 1 --2- 172.21.15.135:0/2654090830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce3c0af750 0x7fce3c0cfce0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fce38009870 tx=0x7fce3802b980 comp rx=0 tx=0).stop 2024-04-17T06:09:39.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7faf44c35640 1 Processor -- start 2024-04-17T06:09:39.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7faf44c35640 1 -- start start 2024-04-17T06:09:39.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7fce43667640 1 -- 172.21.15.135:0/2654090830 >> 172.21.15.135:0/2654090830 conn(0x7fce3c0cd7c0 msgr2=0x7fce3c0cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.234+0000 7fee15cd9640 1 -- 172.21.15.135:0/484892291 wait complete. 2024-04-17T06:09:39.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.235+0000 7fe7952e4640 1 Processor -- start 2024-04-17T06:09:39.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.235+0000 7faf44c35640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 0x7faf380d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.235+0000 7fee15cd9640 1 Processor -- start 2024-04-17T06:09:39.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.235+0000 7fe7952e4640 1 -- start start 2024-04-17T06:09:39.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf44c35640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf38038c80 0x7faf3804ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7f9f3279d640 1 Processor -- start 2024-04-17T06:09:39.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf3dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf38038c80 0x7faf3804ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf3dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf38038c80 0x7faf3804ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48680/0 (socket says 172.21.15.135:48680) 2024-04-17T06:09:39.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf3e59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 0x7faf380d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.252 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf3dd9b640 1 -- 172.21.15.135:0/1365502263 learned_addr learned my addr 172.21.15.135:0/1365502263 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7fee15cd9640 1 -- start start 2024-04-17T06:09:39.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf44c35640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf380e1200 0x7faf380504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf44c35640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf380524a0 con 0x7faf38038c80 2024-04-17T06:09:39.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7f9f3279d640 1 -- start start 2024-04-17T06:09:39.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf44c35640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faf38051890 con 0x7faf380e1200 2024-04-17T06:09:39.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf44c35640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faf38054260 con 0x7faf38036120 2024-04-17T06:09:39.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7faf3ed9d640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf380e1200 0x7faf380504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7fe7952e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7880dfee0 0x7fe7880d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7fe7952e4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7880e40f0 0x7fe788033bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7fe7952e4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe788051920 0x7fe788034130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7fe7952e4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe788056b40 con 0x7fe7880dfee0 2024-04-17T06:09:39.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7fe7952e4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe788009130 con 0x7fe7880e40f0 2024-04-17T06:09:39.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.236+0000 7fe7952e4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe7880e4900 con 0x7fe788051920 2024-04-17T06:09:39.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fe78effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7880dfee0 0x7fe7880d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7faf3dd9b640 1 -- 172.21.15.135:0/1365502263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 msgr2=0x7faf380d44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fce43667640 1 -- 172.21.15.135:0/2654090830 shutdown_connections 2024-04-17T06:09:39.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7faf3dd9b640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf38036120 0x7faf380d44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7faf3dd9b640 1 -- 172.21.15.135:0/1365502263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf380e1200 msgr2=0x7faf380504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7faf3dd9b640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf380e1200 0x7faf380504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7faf3dd9b640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3804acf0 con 0x7faf38038c80 2024-04-17T06:09:39.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fe78effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7880dfee0 0x7fe7880d1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48690/0 (socket says 172.21.15.135:48690) 2024-04-17T06:09:39.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fe78effd640 1 -- 172.21.15.135:0/3144534004 learned_addr learned my addr 172.21.15.135:0/3144534004 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fce43667640 1 -- 172.21.15.135:0/2654090830 wait complete. 2024-04-17T06:09:39.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7faf3dd9b640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf38038c80 0x7faf3804ff60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf3000d920 tx=0x7faf3000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7f9f3279d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 0x7f9f2c0cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fe78effd640 1 -- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe788051920 msgr2=0x7fe788034130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T06:09:39.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fe78effd640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe788051920 0x7fe788034130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fe78effd640 1 -- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7880e40f0 msgr2=0x7fe788033bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fe78effd640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7880e40f0 0x7fe788033bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7fe78effd640 1 -- 172.21.15.135:0/3144534004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7880528c0 con 0x7fe7880dfee0 2024-04-17T06:09:39.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.237+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3000b780 con 0x7faf38038c80 2024-04-17T06:09:39.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 0x7f9f2c0cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf30019070 con 0x7faf38038c80 2024-04-17T06:09:39.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7faf44c35640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3804afd0 con 0x7faf38038c80 2024-04-17T06:09:39.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3000bb40 con 0x7faf38038c80 2024-04-17T06:09:39.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7fe78effd640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7880dfee0 0x7fe7880d1e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c530 tx=0x7fe78400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 0x7f9f2c0cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48704/0 (socket says 172.21.15.135:48704) 2024-04-17T06:09:39.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f2bfff640 1 -- 172.21.15.135:0/2004411824 learned_addr learned my addr 172.21.15.135:0/2004411824 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f3279d640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c0b37c0 0x7f9f2c1fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f3279d640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c167600 0x7f9f2c0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c06e980 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c1640b0 con 0x7f9f2c0b37c0 2024-04-17T06:09:39.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c06eb00 con 0x7f9f2c167600 2024-04-17T06:09:39.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7fee15cd9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee08006e90 0x7fee0804bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7fee15cd9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee0804ffc0 0x7fee0804daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f2bfff640 1 -- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c167600 msgr2=0x7f9f2c0ad670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T06:09:39.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7fee0f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee08006e90 0x7fee0804bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7fee0f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee08006e90 0x7fee0804bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48708/0 (socket says 172.21.15.135:48708) 2024-04-17T06:09:39.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f2bfff640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f2c167600 0x7f9f2c0ad670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f2bfff640 1 -- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c0b37c0 msgr2=0x7f9f2c1fa080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T06:09:39.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f2bfff640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f2c0b37c0 0x7f9f2c1fa080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.238+0000 7f9f2bfff640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c1fa8d0 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee0f7fe640 1 -- 172.21.15.135:0/2560806177 learned_addr learned my addr 172.21.15.135:0/2560806177 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee15cd9640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee080eacf0 0x7fee0804e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fce43667640 1 Processor -- start 2024-04-17T06:09:39.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee15cd9640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fee08054930 con 0x7fee08006e90 2024-04-17T06:09:39.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee15cd9640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fee080547b0 con 0x7fee080eacf0 2024-04-17T06:09:39.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee15cd9640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fee0804f540 con 0x7fee0804ffc0 2024-04-17T06:09:39.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7f9f2bfff640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 0x7f9f2c0cbf90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c02b950 tx=0x7f9f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fce43667640 1 -- start start 2024-04-17T06:09:39.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee0effd640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee0804ffc0 0x7fee0804daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee0ffff640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee080eacf0 0x7fee0804e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784017070 con 0x7fe7880dfee0 2024-04-17T06:09:39.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe784005ce0 con 0x7fe7880dfee0 2024-04-17T06:09:39.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee0effd640 1 -- 172.21.15.135:0/2560806177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee080eacf0 msgr2=0x7fee0804e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe788050e40 con 0x7fe7880dfee0 2024-04-17T06:09:39.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee0effd640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee080eacf0 0x7fee0804e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe788051400 con 0x7fe7880dfee0 2024-04-17T06:09:39.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784007290 con 0x7fe7880dfee0 2024-04-17T06:09:39.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee0effd640 1 -- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee08006e90 msgr2=0x7fee0804bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee0effd640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee08006e90 0x7fee0804bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7fee0effd640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0804e570 con 0x7fee0804ffc0 2024-04-17T06:09:39.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce43667640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce3c06bed0 0x7fce3c151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce43667640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce43667640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce3c152810 0x7fce3c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce43667640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c0b03f0 con 0x7fce3c152810 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce43667640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c0b5c20 con 0x7fce3c06bed0 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce43667640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c06c320 con 0x7fce3c0afd00 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce40c02640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce40c02640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:49876/0 (socket says 172.21.15.135:49876) 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce40c02640 1 -- 172.21.15.135:0/484950173 learned_addr learned my addr 172.21.15.135:0/484950173 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce41c04640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce3c152810 0x7fce3c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce40c02640 1 -- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce3c06bed0 msgr2=0x7fce3c151f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce40c02640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce3c06bed0 0x7fce3c151f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce40c02640 1 -- 172.21.15.135:0/484950173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce3c152810 msgr2=0x7fce3c06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce40c02640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fce3c152810 0x7fce3c06ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7fce40c02640 1 -- 172.21.15.135:0/484950173 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c1524c0 con 0x7fce3c0afd00 2024-04-17T06:09:39.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7f423291c640 1 Processor -- start 2024-04-17T06:09:39.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.240+0000 7f423291c640 1 -- start start 2024-04-17T06:09:39.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fce40c02640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06e780 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c7f0 tx=0x7fce2c00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fee0effd640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee0804ffc0 0x7fee0804daf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee0400c390 tx=0x7fee0400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c015070 con 0x7fce3c0afd00 2024-04-17T06:09:39.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c004220 con 0x7fce3c0afd00 2024-04-17T06:09:39.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c007620 con 0x7fce3c0afd00 2024-04-17T06:09:39.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fce43667640 1 -- 172.21.15.135:0/484950173 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c157390 con 0x7fce3c0afd00 2024-04-17T06:09:39.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04017070 con 0x7fee0804ffc0 2024-04-17T06:09:39.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee040052d0 con 0x7fee0804ffc0 2024-04-17T06:09:39.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7fce43667640 1 -- 172.21.15.135:0/484950173 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c06fda0 con 0x7fce3c0afd00 2024-04-17T06:09:39.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04003830 con 0x7fee0804ffc0 2024-04-17T06:09:39.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7fee15cd9640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0804e7e0 con 0x7fee0804ffc0 2024-04-17T06:09:39.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7fee15cd9640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fee08034770 con 0x7fee0804ffc0 2024-04-17T06:09:39.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c040070 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7f423291c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f42240091c0 0x7f4224052f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7f423291c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42240ec0f0 con 0x7f42240490a0 2024-04-17T06:09:39.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7f423291c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42240095a0 con 0x7f4224048c70 2024-04-17T06:09:39.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7f423291c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f42240359a0 con 0x7f42240091c0 2024-04-17T06:09:39.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7f4230eb9640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f42240490a0 0x7f42240eb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37734/0 (socket says 172.21.15.135:37734) 2024-04-17T06:09:39.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7f4230eb9640 1 -- 172.21.15.135:0/3432545942 learned_addr learned my addr 172.21.15.135:0/3432545942 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:39.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7f422bfff640 1 --2- 172.21.15.135:0/3432545942 >> v2:172.21.15.190:3300/0 conn(0x7f42240091c0 0x7f4224052f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7f422affd640 1 -- 172.21.15.135:0/3432545942 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4102945217 0 0) 0x7f42240ec0f0 con 0x7f42240490a0 2024-04-17T06:09:39.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7f422affd640 1 -- 172.21.15.135:0/3432545942 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f421c003520 con 0x7f42240490a0 2024-04-17T06:09:39.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.239+0000 7faf44c35640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3804b2b0 con 0x7faf38038c80 2024-04-17T06:09:39.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.241+0000 7faf44c35640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf380057d0 con 0x7faf38038c80 2024-04-17T06:09:39.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf3000bce0 con 0x7faf38038c80 2024-04-17T06:09:39.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.242+0000 7faf2effd640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf2006c8f0 0x7faf2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf3008bc30 con 0x7faf38038c80 2024-04-17T06:09:39.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7faf2effd640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6814/4143480345,v1:172.21.15.135:6817/4143480345] conn(0x7faf20075fe0 0x7faf20078420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.135:6814/4143480345,v1:172.21.15.135:6817/4143480345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf20078af0 con 0x7faf20075fe0 2024-04-17T06:09:39.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf3008bf30 con 0x7faf38038c80 2024-04-17T06:09:39.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7f422bfff640 1 -- 172.21.15.135:0/3432545942 >> v1:172.21.15.190:6789/0 conn(0x7f4224048c70 legacy=0x7f42240eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7f422affd640 1 -- 172.21.15.135:0/3432545942 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2861056205 0 0) 0x7f421c003520 con 0x7f42240490a0 2024-04-17T06:09:39.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7fee15cd9640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee08005380 con 0x7fee0804ffc0 2024-04-17T06:09:39.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee0401a070 con 0x7fee0804ffc0 2024-04-17T06:09:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7f422bfff640 1 -- 172.21.15.135:0/3432545942 >> v1:172.21.15.135:6789/0 conn(0x7f42240490a0 legacy=0x7f42240eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7f422bfff640 1 -- 172.21.15.135:0/3432545942 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4224035c60 con 0x7f42240091c0 2024-04-17T06:09:39.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7fce43667640 1 -- 172.21.15.135:0/484950173 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce3c068a20 con 0x7fce3c0afd00 2024-04-17T06:09:39.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce2c0079d0 con 0x7fce3c0afd00 2024-04-17T06:09:39.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7fce31ffb640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce2006c9c0 0x7fce2006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe78401a070 con 0x7fe7880dfee0 2024-04-17T06:09:39.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7fe7777fe640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe77c06c9c0 0x7fe77c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7fe78e7fc640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe77c06c9c0 0x7fe77c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1c030d00 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c040070 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c1fabb0 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c1fb170 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.243+0000 7faf3ed9d640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6814/4143480345,v1:172.21.15.135:6817/4143480345] conn(0x7faf20075fe0 0x7faf20078420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.244+0000 7faf3ed9d640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6814/4143480345,v1:172.21.15.135:6817/4143480345] conn(0x7faf20075fe0 0x7faf20078420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7fedfbfff640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fedf406c9c0 0x7fedf406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe770001030 con 0x7fe7880dfee0 2024-04-17T06:09:39.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fee0408b130 con 0x7fee0804ffc0 2024-04-17T06:09:39.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7fce41403640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce2006c9c0 0x7fce2006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7fe78e7fc640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe77c06c9c0 0x7fe77c06ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe78800cdd0 tx=0x7fe780008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f2c074940 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7fedfbfff640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6808/2300683224,v1:172.21.15.135:6809/2300683224] conn(0x7fedf40760c0 0x7fedf4078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.135:6808/2300683224,v1:172.21.15.135:6809/2300683224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedf4078bd0 con 0x7fedf40760c0 2024-04-17T06:09:39.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.245+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fee04092050 con 0x7fee0804ffc0 2024-04-17T06:09:39.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fee0f7fe640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fedf406c9c0 0x7fedf406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fce2c08c150 con 0x7fce3c0afd00 2024-04-17T06:09:39.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fce41403640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce2006c9c0 0x7fce2006ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce38036000 tx=0x7fce3803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fee0ffff640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6808/2300683224,v1:172.21.15.135:6809/2300683224] conn(0x7fedf40760c0 0x7fedf4078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fce31ffb640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:6800/4084657614,v1:172.21.15.190:6801/4084657614] conn(0x7fce200760c0 0x7fce20078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 --> [v2:172.21.15.190:6800/4084657614,v1:172.21.15.190:6801/4084657614] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce2c01be50 con 0x7fce200760c0 2024-04-17T06:09:39.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fce2c093050 con 0x7fce3c0afd00 2024-04-17T06:09:39.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fee0ffff640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6808/2300683224,v1:172.21.15.135:6809/2300683224] conn(0x7fedf40760c0 0x7fedf4078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7faf3e59c640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf2006c8f0 0x7faf2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fce41c04640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:6800/4084657614,v1:172.21.15.190:6801/4084657614] conn(0x7fce200760c0 0x7fce20078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7fee0f7fe640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fedf406c9c0 0x7fedf406ee80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee10000b60 tx=0x7fee1006f030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 <== osd.2 v2:172.21.15.135:6814/4143480345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf20078af0 con 0x7faf20075fe0 2024-04-17T06:09:39.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.246+0000 7f422bfff640 1 --2- 172.21.15.135:0/3432545942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42240091c0 0x7f4224052f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f94ac981560818 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7faf3e59c640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf2006c8f0 0x7faf2006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf38037db0 tx=0x7faf34009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7f422affd640 1 -- 172.21.15.135:0/3432545942 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220037070 con 0x7f42240091c0 2024-04-17T06:09:39.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7f422affd640 1 -- 172.21.15.135:0/3432545942 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4220030510 con 0x7f42240091c0 2024-04-17T06:09:39.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7f422affd640 1 -- 172.21.15.135:0/3432545942 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c0505a0 con 0x7f42240091c0 2024-04-17T06:09:39.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe78408b130 con 0x7fe7880dfee0 2024-04-17T06:09:39.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7fce41c04640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:6800/4084657614,v1:172.21.15.190:6801/4084657614] conn(0x7fce200760c0 0x7fce20078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7fe7777fe640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:6816/1825459581,v1:172.21.15.190:6817/1825459581] conn(0x7fe77c0760b0 0x7fe77c0784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f1c04b060 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 --> [v2:172.21.15.190:6816/1825459581,v1:172.21.15.190:6817/1825459581] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe77c078bc0 con 0x7fe77c0760b0 2024-04-17T06:09:39.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 <== osd.0 v2:172.21.15.135:6808/2300683224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedf4078bd0 con 0x7fedf40760c0 2024-04-17T06:09:39.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe784092050 con 0x7fe7880dfee0 2024-04-17T06:09:39.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.247+0000 7fe78f7fe640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:6816/1825459581,v1:172.21.15.190:6817/1825459581] conn(0x7fe77c0760b0 0x7fe77c0784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f9f28ff9640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f1806caf0 0x7f9f1806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9f1c0ad7d0 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f9f2b7fe640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f1806caf0 0x7f9f1806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f9f28ff9640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:6820/2045080651,v1:172.21.15.190:6822/2045080651] conn(0x7f9f180761f0 0x7f9f18078630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7fe78f7fe640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:6816/1825459581,v1:172.21.15.190:6817/1825459581] conn(0x7fe77c0760b0 0x7fe77c0784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.190:6820/2045080651,v1:172.21.15.190:6822/2045080651] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f1c04ae50 con 0x7f9f180761f0 2024-04-17T06:09:39.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9f1c0b22c0 con 0x7f9f2c0afbe0 2024-04-17T06:09:39.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f423291c640 1 -- 172.21.15.135:0/3432545942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42240091c0 msgr2=0x7f4224052f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f423291c640 1 --2- 172.21.15.135:0/3432545942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42240091c0 0x7f4224052f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002b790 comp rx=0 tx=0).stop 2024-04-17T06:09:39.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 <== osd.4 v2:172.21.15.190:6800/4084657614 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce2c01be50 con 0x7fce200760c0 2024-04-17T06:09:39.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f423291c640 1 -- 172.21.15.135:0/3432545942 shutdown_connections 2024-04-17T06:09:39.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f423291c640 1 --2- 172.21.15.135:0/3432545942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42240091c0 0x7f4224052f30 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002b790 comp rx=0 tx=0).stop 2024-04-17T06:09:39.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f423291c640 1 -- 172.21.15.135:0/3432545942 >> 172.21.15.135:0/3432545942 conn(0x7f422401ef70 msgr2=0x7f4224055ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f9f30d3a640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:6820/2045080651,v1:172.21.15.190:6822/2045080651] conn(0x7f9f180761f0 0x7f9f18078630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f423291c640 1 -- 172.21.15.135:0/3432545942 shutdown_connections 2024-04-17T06:09:39.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.248+0000 7f423291c640 1 -- 172.21.15.135:0/3432545942 wait complete. 2024-04-17T06:09:39.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.249+0000 7f9f2b7fe640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f1806caf0 0x7f9f1806efb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9f20006fd0 tx=0x7f9f20008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.249+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 <== osd.5 v2:172.21.15.190:6816/1825459581 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe77c078bc0 con 0x7fe77c0760b0 2024-04-17T06:09:39.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.250+0000 7f423291c640 1 Processor -- start 2024-04-17T06:09:39.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.250+0000 7f9f30d3a640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:6820/2045080651,v1:172.21.15.190:6822/2045080651] conn(0x7f9f180761f0 0x7f9f18078630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.250+0000 7f423291c640 1 -- start start 2024-04-17T06:09:39.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.251+0000 7f89c7077640 1 Processor -- start 2024-04-17T06:09:39.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.251+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 <== osd.6 v2:172.21.15.190:6820/2045080651 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f1c04ae50 con 0x7f9f180761f0 2024-04-17T06:09:39.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.251+0000 7f423291c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4224048c70 0x7f42240daf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.251+0000 7f423291c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42240490a0 0x7f42240332f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.252+0000 7f422b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42240490a0 0x7f42240332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.252+0000 7f422b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42240490a0 0x7f42240332f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48728/0 (socket says 172.21.15.135:48728) 2024-04-17T06:09:39.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.252+0000 7f422b7fe640 1 -- 172.21.15.135:0/830861369 learned_addr learned my addr 172.21.15.135:0/830861369 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.252+0000 7f423291c640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4224033830 0x7f42240e9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.252+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4224036500 con 0x7f42240490a0 2024-04-17T06:09:39.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.252+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4224036220 con 0x7f4224048c70 2024-04-17T06:09:39.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.252+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f42240367e0 con 0x7f4224033830 2024-04-17T06:09:39.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.251+0000 7f89c7077640 1 -- start start 2024-04-17T06:09:39.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f89c7077640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f89b80eacf0 0x7f89b804fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f89c7077640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f89b804ffc0 0x7f89b8037bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f89c7077640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b8053710 con 0x7f89b8038100 2024-04-17T06:09:39.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f89c7077640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f89b8054320 con 0x7f89b804ffc0 2024-04-17T06:09:39.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f89c7077640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f89b80539f0 con 0x7f89b80eacf0 2024-04-17T06:09:39.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f89c6876640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f89b8038100 0x7f89b8038570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39326/0 (socket says 172.21.15.135:39326) 2024-04-17T06:09:39.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f89c6876640 1 -- 172.21.15.135:0/1628951313 learned_addr learned my addr 172.21.15.135:0/1628951313 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:39.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f89c5874640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f89b804ffc0 0x7f89b8037bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f4230eb9640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4224033830 0x7f42240e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f422bfff640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4224048c70 0x7f42240daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f422b7fe640 1 -- 172.21.15.135:0/830861369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4224033830 msgr2=0x7f42240e9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f422b7fe640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4224033830 0x7f42240e9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f422b7fe640 1 -- 172.21.15.135:0/830861369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4224048c70 msgr2=0x7f42240daf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f422b7fe640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4224048c70 0x7f42240daf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7f422b7fe640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4224050370 con 0x7f42240490a0 2024-04-17T06:09:39.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.251+0000 7fb79b493640 1 Processor -- start 2024-04-17T06:09:39.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.251+0000 7fb79b493640 1 -- start start 2024-04-17T06:09:39.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7fb79b493640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb7940b3d50 0x7fb79415b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7fb79b493640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7940b05c0 con 0x7fb7940b01e0 2024-04-17T06:09:39.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7fb79b493640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7940b4310 con 0x7fb79406c9a0 2024-04-17T06:09:39.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7fb79b493640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb7940b45f0 con 0x7fb7940b3d50 2024-04-17T06:09:39.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7fb799a30640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fb79406c9a0 0x7fb7940ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39336/0 (socket says 172.21.15.135:39336) 2024-04-17T06:09:39.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.253+0000 7fb799a30640 1 -- 172.21.15.135:0/748226063 learned_addr learned my addr 172.21.15.135:0/748226063 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:39.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7f89c6075640 1 --2- 172.21.15.135:0/1628951313 >> v2:172.21.15.190:3300/0 conn(0x7f89b80eacf0 0x7f89b804fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7fb798a2e640 1 --2- 172.21.15.135:0/748226063 >> v2:172.21.15.135:3300/0 conn(0x7fb7940b3d50 0x7fb79415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7f89c5874640 1 -- 172.21.15.135:0/1628951313 >> v2:172.21.15.190:3300/0 conn(0x7f89b80eacf0 msgr2=0x7f89b804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7f89c5874640 1 --2- 172.21.15.135:0/1628951313 >> v2:172.21.15.190:3300/0 conn(0x7f89b80eacf0 0x7f89b804fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7f89c5874640 1 -- 172.21.15.135:0/1628951313 >> v1:172.21.15.190:6789/0 conn(0x7f89b8038100 legacy=0x7f89b8038570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7fb798a2e640 1 -- 172.21.15.135:0/748226063 >> v1:172.21.15.190:6789/0 conn(0x7fb79406c9a0 legacy=0x7fb7940ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7fb798a2e640 1 -- 172.21.15.135:0/748226063 >> v1:172.21.15.135:6789/0 conn(0x7fb7940b01e0 legacy=0x7fb7940b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7fb798a2e640 1 -- 172.21.15.135:0/748226063 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7940b48d0 con 0x7fb7940b3d50 2024-04-17T06:09:39.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7f89c5874640 1 -- 172.21.15.135:0/1628951313 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b8053cd0 con 0x7f89b804ffc0 2024-04-17T06:09:39.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.254+0000 7f422b7fe640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42240490a0 0x7f42240332f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f421800c780 tx=0x7f421800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.255+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218016020 con 0x7f42240490a0 2024-04-17T06:09:39.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.255+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4224050650 con 0x7f42240490a0 2024-04-17T06:09:39.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.255+0000 7fb798a2e640 1 --2- 172.21.15.135:0/748226063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940b3d50 0x7fb79415b480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb78400b9f0 tx=0x7fb78402dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc2542a7f3f6c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.255+0000 7f89c5874640 1 --2- 172.21.15.135:0/1628951313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 0x7f89b8037bc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89b8007a50 tx=0x7f89b002b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188c7df37d105db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.255+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421800a840 con 0x7f42240490a0 2024-04-17T06:09:39.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.256+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421800ac70 con 0x7f42240490a0 2024-04-17T06:09:39.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.256+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4224050c10 con 0x7f42240490a0 2024-04-17T06:09:39.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.257+0000 7f89c5073640 1 -- 172.21.15.135:0/1628951313 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0037070 con 0x7f89b804ffc0 2024-04-17T06:09:39.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.257+0000 7f89c5073640 1 -- 172.21.15.135:0/1628951313 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89b0030710 con 0x7f89b804ffc0 2024-04-17T06:09:39.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.257+0000 7f89c5073640 1 -- 172.21.15.135:0/1628951313 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0030ae0 con 0x7f89b804ffc0 2024-04-17T06:09:39.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f89c7077640 1 -- 172.21.15.135:0/1628951313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 msgr2=0x7f89b8037bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f89c7077640 1 --2- 172.21.15.135:0/1628951313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 0x7f89b8037bc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89b8007a50 tx=0x7f89b002b6f0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f89c7077640 1 -- 172.21.15.135:0/1628951313 shutdown_connections 2024-04-17T06:09:39.324 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f89c7077640 1 --2- 172.21.15.135:0/1628951313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 0x7f89b8037bc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f89c7077640 1 --2- 172.21.15.135:0/1628951313 >> v2:172.21.15.190:3300/0 conn(0x7f89b80eacf0 0x7f89b804fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f89c7077640 1 -- 172.21.15.135:0/1628951313 >> 172.21.15.135:0/1628951313 conn(0x7f89b8009fa0 msgr2=0x7f89b800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f420c001030 con 0x7f42240490a0 2024-04-17T06:09:39.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.257+0000 7fb78bfff640 1 -- 172.21.15.135:0/748226063 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784034070 con 0x7fb7940b3d50 2024-04-17T06:09:39.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.257+0000 7fb78bfff640 1 -- 172.21.15.135:0/748226063 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78402e9e0 con 0x7fb7940b3d50 2024-04-17T06:09:39.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7fb79b493640 1 -- 172.21.15.135:0/748226063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940b3d50 msgr2=0x7fb79415b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7fb79b493640 1 --2- 172.21.15.135:0/748226063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940b3d50 0x7fb79415b480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb78400b9f0 tx=0x7fb78402dc30 comp rx=0 tx=0).stop 2024-04-17T06:09:39.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7fb79b493640 1 -- 172.21.15.135:0/748226063 shutdown_connections 2024-04-17T06:09:39.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7fb79b493640 1 --2- 172.21.15.135:0/748226063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940b3d50 0x7fb79415b480 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7fb79b493640 1 -- 172.21.15.135:0/748226063 >> 172.21.15.135:0/748226063 conn(0x7fb794072780 msgr2=0x7fb794072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4218019020 con 0x7f42240490a0 2024-04-17T06:09:39.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f89c7077640 1 -- 172.21.15.135:0/1628951313 shutdown_connections 2024-04-17T06:09:39.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7f89c7077640 1 -- 172.21.15.135:0/1628951313 wait complete. 2024-04-17T06:09:39.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.259+0000 7f4228ff9640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f421006c9c0 0x7f421006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.259+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f421808be00 con 0x7f42240490a0 2024-04-17T06:09:39.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.259+0000 7f4228ff9640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6821/4267051868,v1:172.21.15.135:6822/4267051868] conn(0x7f4210076140 0x7f4210078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.259+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.135:6821/4267051868,v1:172.21.15.135:6822/4267051868] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4210078c50 con 0x7f4210076140 2024-04-17T06:09:39.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.259+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f421808e460 con 0x7f42240490a0 2024-04-17T06:09:39.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.259+0000 7f422bfff640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f421006c9c0 0x7f421006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7fb79b493640 1 -- 172.21.15.135:0/748226063 shutdown_connections 2024-04-17T06:09:39.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.260+0000 7f4230eb9640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6821/4267051868,v1:172.21.15.135:6822/4267051868] conn(0x7f4210076140 0x7f4210078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.260+0000 7f89c7077640 1 Processor -- start 2024-04-17T06:09:39.331 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.260+0000 7f89c7077640 1 -- start start 2024-04-17T06:09:39.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.260+0000 7f4230eb9640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6821/4267051868,v1:172.21.15.135:6822/4267051868] conn(0x7f4210076140 0x7f4210078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.261+0000 7f422bfff640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f421006c9c0 0x7f421006ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4220034000 tx=0x7f422002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.261+0000 7f89c7077640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b8038100 0x7f89b8036630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.261+0000 7f89c7077640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 0x7f89b80dbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.261+0000 7f89c7077640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b80eacf0 0x7f89b80dcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 <== osd.3 v2:172.21.15.135:6821/4267051868 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4210078c50 con 0x7f4210076140 2024-04-17T06:09:39.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c6075640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b8038100 0x7f89b8036630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c6075640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b8038100 0x7f89b8036630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36498/0 (socket says 172.21.15.135:36498) 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c6075640 1 -- 172.21.15.135:0/784492069 learned_addr learned my addr 172.21.15.135:0/784492069 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c6876640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b80eacf0 0x7f89b80dcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c7077640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89b80547b0 con 0x7f89b804ffc0 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c7077640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f89b8038c80 con 0x7f89b8038100 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c7077640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f89b8054930 con 0x7f89b80eacf0 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c6075640 1 -- 172.21.15.135:0/784492069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b80eacf0 msgr2=0x7f89b80dcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c6075640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89b80eacf0 0x7f89b80dcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7f89c6075640 1 -- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 msgr2=0x7f89b80dbf90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7f89c5874640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 0x7f89b80dbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7f89c6075640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 0x7f89b80dbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7f89c6075640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b8035590 con 0x7f89b8038100 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7f89c5874640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89b804ffc0 0x7f89b80dbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7f89c6075640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b8038100 0x7f89b8036630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00d5f0 tx=0x7f89bc00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.264+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0125f0 con 0x7f89b8038100 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.264+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc019070 con 0x7f89b8038100 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.264+0000 7f89c7077640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b8035870 con 0x7f89b8038100 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.264+0000 7f89c7077640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b8035e30 con 0x7f89b8038100 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.267+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc0129b0 con 0x7f89b8038100 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.258+0000 7fb79b493640 1 -- 172.21.15.135:0/748226063 wait complete. 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.260+0000 7fb79b493640 1 Processor -- start 2024-04-17T06:09:39.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.260+0000 7fb79b493640 1 -- start start 2024-04-17T06:09:39.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.268+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89a8001030 con 0x7f89b8038100 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.261+0000 7fb79b493640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406c9a0 0x7fb794153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.261+0000 7fb79b493640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940b01e0 0x7fb7940b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.261+0000 7fb79b493640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940ce1a0 0x7fb79409cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb799a30640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940ce1a0 0x7fb79409cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb799a30640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940ce1a0 0x7fb79409cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48750/0 (socket says 172.21.15.135:48750) 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb79b493640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940cd550 con 0x7fb7940ce1a0 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb79b493640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940b4e90 con 0x7fb79406c9a0 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb79b493640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb79406ce10 con 0x7fb7940b01e0 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb799a30640 1 -- 172.21.15.135:0/2305874306 learned_addr learned my addr 172.21.15.135:0/2305874306 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb799a30640 1 -- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940b01e0 msgr2=0x7fb7940b1d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:09:39.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb799a30640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7940b01e0 0x7fb7940b1d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb799a30640 1 -- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406c9a0 msgr2=0x7fb794153aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb799a30640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb79406c9a0 0x7fb794153aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.262+0000 7fb799a30640 1 -- 172.21.15.135:0/2305874306 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb794154960 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7fb799a30640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940ce1a0 0x7fb79409cc80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00cbf0 tx=0x7fb78c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c018070 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c004b60 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.263+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00ab40 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.264+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7941543a0 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.264+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7940cea10 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.267+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb78c01b070 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.267+0000 7fb789ffb640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb7780b0a00 0x7fb7780b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.268+0000 7fb7777fe640 1 -- 172.21.15.135:0/2305874306 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb794068a20 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.268+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb78c08b7c0 con 0x7fb7940ce1a0 2024-04-17T06:09:39.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.268+0000 7fb789ffb640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:6808/163146865,v1:172.21.15.190:6809/163146865] conn(0x7fb7780ba100 0x7fb7780bc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.268+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 --> [v2:172.21.15.190:6808/163146865,v1:172.21.15.190:6809/163146865] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7780bcc10 con 0x7fb7780ba100 2024-04-17T06:09:39.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.268+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb78c055430 con 0x7fb7940ce1a0 2024-04-17T06:09:39.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.269+0000 7fb79922f640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb7780b0a00 0x7fb7780b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.269+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89bc012b50 con 0x7f89b8038100 2024-04-17T06:09:39.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.269+0000 7fb79922f640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb7780b0a00 0x7fb7780b2ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb790009620 tx=0x7fb790006b30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.270+0000 7fb798a2e640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:6808/163146865,v1:172.21.15.190:6809/163146865] conn(0x7fb7780ba100 0x7fb7780bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.270+0000 7f89b6ffd640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f89a406c9c0 0x7f89a406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.270+0000 7fb798a2e640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:6808/163146865,v1:172.21.15.190:6809/163146865] conn(0x7fb7780ba100 0x7fb7780bc540 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.270+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f89bc08c280 con 0x7f89b8038100 2024-04-17T06:09:39.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.270+0000 7f89c5874640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f89a406c9c0 0x7f89a406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.271+0000 7f89b6ffd640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6800/1452262953,v1:172.21.15.135:6801/1452262953] conn(0x7f89a4076140 0x7f89a4078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.271+0000 7f89c5874640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f89a406c9c0 0x7f89a406ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89b0034000 tx=0x7f89b0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.271+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 <== osd.7 v2:172.21.15.190:6808/163146865 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7780bcc10 con 0x7fb7780ba100 2024-04-17T06:09:39.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.271+0000 7f89c6876640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6800/1452262953,v1:172.21.15.135:6801/1452262953] conn(0x7f89a4076140 0x7f89a4078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.272+0000 7f89c6876640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6800/1452262953,v1:172.21.15.135:6801/1452262953] conn(0x7f89a4076140 0x7f89a4078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.272+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.135:6800/1452262953,v1:172.21.15.135:6801/1452262953] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89bc001d30 con 0x7f89a4076140 2024-04-17T06:09:39.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.272+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f89bc093050 con 0x7f89b8038100 2024-04-17T06:09:39.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.273+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 <== osd.1 v2:172.21.15.135:6800/1452262953 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89bc001d30 con 0x7f89a4076140 2024-04-17T06:09:39.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.278+0000 7fee15cd9640 1 -- 172.21.15.135:0/2560806177 --> [v2:172.21.15.135:6808/2300683224,v1:172.21.15.135:6809/2300683224] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee08047a60 con 0x7fedf40760c0 2024-04-17T06:09:39.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.278+0000 7fedfbfff640 1 -- 172.21.15.135:0/2560806177 <== osd.0 v2:172.21.15.135:6808/2300683224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fee08047a60 con 0x7fedf40760c0 2024-04-17T06:09:39.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.279+0000 7faf44c35640 1 -- 172.21.15.135:0/1365502263 --> [v2:172.21.15.135:6814/4143480345,v1:172.21.15.135:6817/4143480345] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf380e15e0 con 0x7faf20075fe0 2024-04-17T06:09:39.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.279+0000 7fedf9ffb640 1 -- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6808/2300683224,v1:172.21.15.135:6809/2300683224] conn(0x7fedf40760c0 msgr2=0x7fedf4078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.279+0000 7fedf9ffb640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6808/2300683224,v1:172.21.15.135:6809/2300683224] conn(0x7fedf40760c0 0x7fedf4078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.279+0000 7fedf9ffb640 1 -- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fedf406c9c0 msgr2=0x7fedf406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.279+0000 7fedf9ffb640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fedf406c9c0 0x7fedf406ee80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee10000b60 tx=0x7fee1006f030 comp rx=0 tx=0).stop 2024-04-17T06:09:39.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.279+0000 7fedf9ffb640 1 -- 172.21.15.135:0/2560806177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee0804ffc0 msgr2=0x7fee0804daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.279+0000 7fedf9ffb640 1 --2- 172.21.15.135:0/2560806177 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee0804ffc0 0x7fee0804daf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee0400c390 tx=0x7fee0400c850 comp rx=0 tx=0).stop 2024-04-17T06:09:39.347 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.279+0000 7faf2effd640 1 -- 172.21.15.135:0/1365502263 <== osd.2 v2:172.21.15.135:6814/4143480345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faf380e15e0 con 0x7faf20075fe0 2024-04-17T06:09:39.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7faf2cff9640 1 -- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6814/4143480345,v1:172.21.15.135:6817/4143480345] conn(0x7faf20075fe0 msgr2=0x7faf20078420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7faf2cff9640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6814/4143480345,v1:172.21.15.135:6817/4143480345] conn(0x7faf20075fe0 0x7faf20078420 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7faf2cff9640 1 -- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf2006c8f0 msgr2=0x7faf2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7faf2cff9640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf2006c8f0 0x7faf2006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faf38037db0 tx=0x7faf34009040 comp rx=0 tx=0).stop 2024-04-17T06:09:39.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7faf2cff9640 1 -- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf38038c80 msgr2=0x7faf3804ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7faf2cff9640 1 --2- 172.21.15.135:0/1365502263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf38038c80 0x7faf3804ff60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf3000d920 tx=0x7faf3000dde0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.348 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7faf3ed9d640 1 -- 172.21.15.135:0/1365502263 reap_dead start 2024-04-17T06:09:39.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7fee0ffff640 1 -- 172.21.15.135:0/2560806177 reap_dead start 2024-04-17T06:09:39.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7fedf9ffb640 1 -- 172.21.15.135:0/2560806177 shutdown_connections 2024-04-17T06:09:39.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7fedf9ffb640 1 -- 172.21.15.135:0/2560806177 >> 172.21.15.135:0/2560806177 conn(0x7fee08009fa0 msgr2=0x7fee080e7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.280+0000 7fce43667640 1 -- 172.21.15.135:0/484950173 --> [v2:172.21.15.190:6800/4084657614,v1:172.21.15.190:6801/4084657614] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce3c0aefc0 con 0x7fce200760c0 2024-04-17T06:09:39.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.281+0000 7fedf9ffb640 1 -- 172.21.15.135:0/2560806177 shutdown_connections 2024-04-17T06:09:39.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.281+0000 7faf2cff9640 1 -- 172.21.15.135:0/1365502263 shutdown_connections 2024-04-17T06:09:39.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.281+0000 7faf2cff9640 1 -- 172.21.15.135:0/1365502263 >> 172.21.15.135:0/1365502263 conn(0x7faf38053740 msgr2=0x7faf3803a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.350 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.281+0000 7fedf9ffb640 1 -- 172.21.15.135:0/2560806177 wait complete. 2024-04-17T06:09:39.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.281+0000 7faf2cff9640 1 -- 172.21.15.135:0/1365502263 shutdown_connections 2024-04-17T06:09:39.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.281+0000 7faf2cff9640 1 -- 172.21.15.135:0/1365502263 wait complete. 2024-04-17T06:09:39.351 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.281+0000 7fce31ffb640 1 -- 172.21.15.135:0/484950173 <== osd.4 v2:172.21.15.190:6800/4084657614 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fce3c0aefc0 con 0x7fce200760c0 2024-04-17T06:09:39.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce1f7fe640 1 -- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:6800/4084657614,v1:172.21.15.190:6801/4084657614] conn(0x7fce200760c0 msgr2=0x7fce20078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce1f7fe640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:6800/4084657614,v1:172.21.15.190:6801/4084657614] conn(0x7fce200760c0 0x7fce20078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce1f7fe640 1 -- 172.21.15.135:0/484950173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce2006c9c0 msgr2=0x7fce2006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce1f7fe640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce2006c9c0 0x7fce2006ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce38036000 tx=0x7fce3803b040 comp rx=0 tx=0).stop 2024-04-17T06:09:39.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce1f7fe640 1 -- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 msgr2=0x7fce3c06e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce1f7fe640 1 --2- 172.21.15.135:0/484950173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce3c0afd00 0x7fce3c06e780 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c7f0 tx=0x7fce2c00ccb0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 --> [v2:172.21.15.190:6820/2045080651,v1:172.21.15.190:6822/2045080651] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f2c0b1d50 con 0x7f9f180761f0 2024-04-17T06:09:39.352 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 --> [v2:172.21.15.190:6816/1825459581,v1:172.21.15.190:6817/1825459581] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe770002d50 con 0x7fe77c0760b0 2024-04-17T06:09:39.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce41c04640 1 -- 172.21.15.135:0/484950173 reap_dead start 2024-04-17T06:09:39.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce1f7fe640 1 -- 172.21.15.135:0/484950173 shutdown_connections 2024-04-17T06:09:39.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.282+0000 7fce1f7fe640 1 -- 172.21.15.135:0/484950173 >> 172.21.15.135:0/484950173 conn(0x7fce3c0cd7c0 msgr2=0x7fce3c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.283+0000 7fce1f7fe640 1 -- 172.21.15.135:0/484950173 shutdown_connections 2024-04-17T06:09:39.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.283+0000 7fce1f7fe640 1 -- 172.21.15.135:0/484950173 wait complete. 2024-04-17T06:09:39.353 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.283+0000 7fe7777fe640 1 -- 172.21.15.135:0/3144534004 <== osd.5 v2:172.21.15.190:6816/1825459581 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe770002d50 con 0x7fe77c0760b0 2024-04-17T06:09:39.354 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.283+0000 7f9f28ff9640 1 -- 172.21.15.135:0/2004411824 <== osd.6 v2:172.21.15.190:6820/2045080651 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9f2c0b1d50 con 0x7f9f180761f0 2024-04-17T06:09:39.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.283+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:6816/1825459581,v1:172.21.15.190:6817/1825459581] conn(0x7fe77c0760b0 msgr2=0x7fe77c0784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.283+0000 7fe7952e4640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.190:6816/1825459581,v1:172.21.15.190:6817/1825459581] conn(0x7fe77c0760b0 0x7fe77c0784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe77c06c9c0 msgr2=0x7fe77c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7fe7952e4640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe77c06c9c0 0x7fe77c06ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe78800cdd0 tx=0x7fe780008040 comp rx=0 tx=0).stop 2024-04-17T06:09:39.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7880dfee0 msgr2=0x7fe7880d1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7fe7952e4640 1 --2- 172.21.15.135:0/3144534004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7880dfee0 0x7fe7880d1e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c530 tx=0x7fe78400c9f0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7fe78f7fe640 1 -- 172.21.15.135:0/3144534004 reap_dead start 2024-04-17T06:09:39.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:6820/2045080651,v1:172.21.15.190:6822/2045080651] conn(0x7f9f180761f0 msgr2=0x7f9f18078630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7f9f3279d640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.190:6820/2045080651,v1:172.21.15.190:6822/2045080651] conn(0x7f9f180761f0 0x7f9f18078630 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f1806caf0 msgr2=0x7f9f1806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7f9f3279d640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f1806caf0 0x7f9f1806efb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9f20006fd0 tx=0x7f9f20008040 comp rx=0 tx=0).stop 2024-04-17T06:09:39.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 msgr2=0x7f9f2c0cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7f9f3279d640 1 --2- 172.21.15.135:0/2004411824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f2c0afbe0 0x7f9f2c0cbf90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c02b950 tx=0x7f9f1c0046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7f9f30d3a640 1 -- 172.21.15.135:0/2004411824 reap_dead start 2024-04-17T06:09:39.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 shutdown_connections 2024-04-17T06:09:39.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 >> 172.21.15.135:0/3144534004 conn(0x7fe78801ef30 msgr2=0x7fe7880d6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.284+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 shutdown_connections 2024-04-17T06:09:39.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.285+0000 7fe7952e4640 1 -- 172.21.15.135:0/3144534004 wait complete. 2024-04-17T06:09:39.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.285+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 shutdown_connections 2024-04-17T06:09:39.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.285+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 >> 172.21.15.135:0/2004411824 conn(0x7f9f2c0b2290 msgr2=0x7f9f2c1624c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.286+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 shutdown_connections 2024-04-17T06:09:39.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.286+0000 7f9f3279d640 1 -- 172.21.15.135:0/2004411824 wait complete. 2024-04-17T06:09:39.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.289+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 --> [v2:172.21.15.135:6821/4267051868,v1:172.21.15.135:6822/4267051868] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f420c002dd0 con 0x7f4210076140 2024-04-17T06:09:39.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.289+0000 7f4228ff9640 1 -- 172.21.15.135:0/830861369 <== osd.3 v2:172.21.15.135:6821/4267051868 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f420c002dd0 con 0x7f4210076140 2024-04-17T06:09:39.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.290+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6821/4267051868,v1:172.21.15.135:6822/4267051868] conn(0x7f4210076140 msgr2=0x7f4210078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.290+0000 7f423291c640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6821/4267051868,v1:172.21.15.135:6822/4267051868] conn(0x7f4210076140 0x7f4210078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.290+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f421006c9c0 msgr2=0x7f421006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.290+0000 7f423291c640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f421006c9c0 0x7f421006ee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4220034000 tx=0x7f422002f000 comp rx=0 tx=0).stop 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.291+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42240490a0 msgr2=0x7f42240332f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.291+0000 7f423291c640 1 --2- 172.21.15.135:0/830861369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42240490a0 0x7f42240332f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f421800c780 tx=0x7f421800cc40 comp rx=0 tx=0).stop 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.291+0000 7f4230eb9640 1 -- 172.21.15.135:0/830861369 reap_dead start 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.291+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 shutdown_connections 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.291+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 >> 172.21.15.135:0/830861369 conn(0x7f422401ef70 msgr2=0x7f422400b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.292+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 shutdown_connections 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.292+0000 7f423291c640 1 -- 172.21.15.135:0/830861369 wait complete. 2024-04-17T06:09:39.360 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.299+0000 7fb7777fe640 1 -- 172.21.15.135:0/2305874306 --> [v2:172.21.15.190:6808/163146865,v1:172.21.15.190:6809/163146865] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7940735b0 con 0x7fb7780ba100 2024-04-17T06:09:39.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.299+0000 7fb789ffb640 1 -- 172.21.15.135:0/2305874306 <== osd.7 v2:172.21.15.190:6808/163146865 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb7940735b0 con 0x7fb7780ba100 2024-04-17T06:09:39.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.300+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:6808/163146865,v1:172.21.15.190:6809/163146865] conn(0x7fb7780ba100 msgr2=0x7fb7780bc540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.300+0000 7fb79b493640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.190:6808/163146865,v1:172.21.15.190:6809/163146865] conn(0x7fb7780ba100 0x7fb7780bc540 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.301+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb7780b0a00 msgr2=0x7fb7780b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.301+0000 7fb79b493640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb7780b0a00 0x7fb7780b2ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb790009620 tx=0x7fb790006b30 comp rx=0 tx=0).stop 2024-04-17T06:09:39.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.301+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940ce1a0 msgr2=0x7fb79409cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.301+0000 7fb79b493640 1 --2- 172.21.15.135:0/2305874306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7940ce1a0 0x7fb79409cc80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00cbf0 tx=0x7fb78c00a430 comp rx=0 tx=0).stop 2024-04-17T06:09:39.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.301+0000 7fb799a30640 1 -- 172.21.15.135:0/2305874306 reap_dead start 2024-04-17T06:09:39.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.301+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 shutdown_connections 2024-04-17T06:09:39.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.301+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 >> 172.21.15.135:0/2305874306 conn(0x7fb794072780 msgr2=0x7fb79406e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.302+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 shutdown_connections 2024-04-17T06:09:39.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.302+0000 7fb79b493640 1 -- 172.21.15.135:0/2305874306 wait complete. 2024-04-17T06:09:39.363 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:39.364 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:39.364 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T06:09:39.365 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T06:09:39.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.305+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 --> [v2:172.21.15.135:6800/1452262953,v1:172.21.15.135:6801/1452262953] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f89a8002dd0 con 0x7f89a4076140 2024-04-17T06:09:39.365 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.306+0000 7f89b6ffd640 1 -- 172.21.15.135:0/784492069 <== osd.1 v2:172.21.15.135:6800/1452262953 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f89a8002dd0 con 0x7f89a4076140 2024-04-17T06:09:39.366 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-17T06:09:39.366 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-17T06:09:39.367 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:39.367 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-17T06:09:39.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6800/1452262953,v1:172.21.15.135:6801/1452262953] conn(0x7f89a4076140 msgr2=0x7f89a4078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6800/1452262953,v1:172.21.15.135:6801/1452262953] conn(0x7f89a4076140 0x7f89a4078580 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f89a406c9c0 msgr2=0x7f89a406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f89a406c9c0 0x7f89a406ee80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f89b0034000 tx=0x7f89b0008030 comp rx=0 tx=0).stop 2024-04-17T06:09:39.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b8038100 msgr2=0x7f89b8036630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 --2- 172.21.15.135:0/784492069 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89b8038100 0x7f89b8036630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00d5f0 tx=0x7f89bc00dab0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89c6876640 1 -- 172.21.15.135:0/784492069 reap_dead start 2024-04-17T06:09:39.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 shutdown_connections 2024-04-17T06:09:39.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 >> 172.21.15.135:0/784492069 conn(0x7f89b8009fa0 msgr2=0x7f89b8006360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.307+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 shutdown_connections 2024-04-17T06:09:39.369 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.308+0000 7f89b4ff9640 1 -- 172.21.15.135:0/784492069 wait complete. 2024-04-17T06:09:39.369 INFO:teuthology.orchestra.run.smithi135.stdout:55834574859 2024-04-17T06:09:39.370 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-17T06:09:39.371 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:39.371 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T06:09:39.371 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-17T06:09:39.372 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-17T06:09:39.372 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:39.373 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T06:09:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.583+0000 7f2d1da58640 1 Processor -- start 2024-04-17T06:09:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.583+0000 7f2d1da58640 1 -- start start 2024-04-17T06:09:39.581 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.583+0000 7f564ade9640 1 Processor -- start 2024-04-17T06:09:39.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.583+0000 7fafaee35640 1 Processor -- start 2024-04-17T06:09:39.582 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.583+0000 7f564ade9640 1 -- start start 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.584+0000 7f2d1da58640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2d18070850 0x7f2d1806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.584+0000 7f2d1da58640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d18070d50 con 0x7f2d180b6ee0 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.584+0000 7f2d1da58640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d180b44c0 con 0x7f2d181609b0 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.584+0000 7f2d1da58640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f2d180b4270 con 0x7f2d18070850 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.583+0000 7fafaee35640 1 -- start start 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.584+0000 7fafaee35640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fafa0005e90 0x7fafa00062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.584+0000 7fafaee35640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa0006810 0x7fafa0006ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7fafaee35640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa0053720 con 0x7fafa00ead00 2024-04-17T06:09:39.583 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7fafaee35640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fafa0054330 con 0x7fafa0005e90 2024-04-17T06:09:39.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7fafaee35640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fafa0053a00 con 0x7fafa0006810 2024-04-17T06:09:39.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f2d1d257640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2d18070850 0x7f2d1806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f2d1d257640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f2d18070850 0x7f2d1806c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48760/0 (socket says 172.21.15.135:48760) 2024-04-17T06:09:39.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f2d1d257640 1 -- 172.21.15.135:0/834941097 learned_addr learned my addr 172.21.15.135:0/834941097 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7fafacbd1640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fafa00ead00 0x7fafa004fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37770/0 (socket says 172.21.15.135:37770) 2024-04-17T06:09:39.584 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7fafacbd1640 1 -- 172.21.15.135:0/1600685081 learned_addr learned my addr 172.21.15.135:0/1600685081 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:39.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f564ade9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f56440b4440 0x7f56440b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f2d1d257640 1 -- 172.21.15.135:0/834941097 >> v1:172.21.15.190:6789/0 conn(0x7f2d181609b0 legacy=0x7f2d180b67d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T06:09:39.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f2d1d257640 1 -- 172.21.15.135:0/834941097 >> v1:172.21.15.135:6789/0 conn(0x7f2d180b6ee0 legacy=0x7f2d18163560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f564ade9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56440b3f90 0x7f564406e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7fafad3d2640 1 --2- 172.21.15.135:0/1600685081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa0006810 0x7fafa0006ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f16e759e640 1 Processor -- start 2024-04-17T06:09:39.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f564ade9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f564406f0d0 con 0x7f56440b3b30 2024-04-17T06:09:39.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f564ade9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f56440b4d60 con 0x7f56440b4440 2024-04-17T06:09:39.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f5649386640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56440b3f90 0x7f564406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.586+0000 7f5649386640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56440b3f90 0x7f564406e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36500/0 (socket says 172.21.15.135:36500) 2024-04-17T06:09:39.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.586+0000 7f5649386640 1 -- 172.21.15.135:0/744845133 learned_addr learned my addr 172.21.15.135:0/744845133 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.586+0000 7f564ade9640 1 -- 172.21.15.135:0/744845133 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f564406f370 con 0x7f56440b3f90 2024-04-17T06:09:39.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.586+0000 7f24dd396640 1 Processor -- start 2024-04-17T06:09:39.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.586+0000 7fafad3d2640 1 -- 172.21.15.135:0/1600685081 >> v2:172.21.15.190:3300/0 conn(0x7fafa0005e90 msgr2=0x7fafa00062d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T06:09:39.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.586+0000 7fafad3d2640 1 --2- 172.21.15.135:0/1600685081 >> v2:172.21.15.190:3300/0 conn(0x7fafa0005e90 0x7fafa00062d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.588 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f16e759e640 1 -- start start 2024-04-17T06:09:39.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.586+0000 7f24dd396640 1 -- start start 2024-04-17T06:09:39.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f5648b85640 1 --2- 172.21.15.135:0/744845133 >> v2:172.21.15.135:3300/0 conn(0x7f56440b4440 0x7f56440b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7fd758b3d640 1 Processor -- start 2024-04-17T06:09:39.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7fafad3d2640 1 -- 172.21.15.135:0/1600685081 >> v1:172.21.15.135:6789/0 conn(0x7fafa00ead00 legacy=0x7fafa004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7fafad3d2640 1 -- 172.21.15.135:0/1600685081 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa0053ce0 con 0x7fafa0006810 2024-04-17T06:09:39.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7fd758b3d640 1 -- start start 2024-04-17T06:09:39.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f16e759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0054b50 0x7f16e0051af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.585+0000 7f2d1d257640 1 -- 172.21.15.135:0/834941097 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d180b47a0 con 0x7f2d18070850 2024-04-17T06:09:39.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f16e659c640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f16e00e12b0 0x7f16e00534f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39354/0 (socket says 172.21.15.135:39354) 2024-04-17T06:09:39.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f642f1d4640 1 Processor -- start 2024-04-17T06:09:39.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f16e659c640 1 -- 172.21.15.135:0/1670321921 learned_addr learned my addr 172.21.15.135:0/1670321921 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:39.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f24dd396640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f24d8067b00 0x7f24d815b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f16e5d9b640 1 --2- 172.21.15.135:0/1670321921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0054b50 0x7f16e0051af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f24dd396640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24d815bdd0 0x7f24d8165be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f24dd396640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24d806c440 con 0x7f24d8067720 2024-04-17T06:09:39.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f24dd396640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f24d80b0800 con 0x7f24d8067b00 2024-04-17T06:09:39.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f24dd396640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f24d806c720 con 0x7f24d815bdd0 2024-04-17T06:09:39.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f16e759e640 1 --2- 172.21.15.135:0/1670321921 >> v2:172.21.15.135:3300/0 conn(0x7f16e0053c00 0x7f16e0052030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e6d9d640 1 --2- 172.21.15.135:0/1670321921 >> v2:172.21.15.135:3300/0 conn(0x7f16e0053c00 0x7f16e0052030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f24d77fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24d815bdd0 0x7f24d8165be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e759e640 1 -- 172.21.15.135:0/1670321921 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e0054fe0 con 0x7f16e00e12b0 2024-04-17T06:09:39.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f24d67fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f24d8067b00 0x7f24d815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f24d77fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24d815bdd0 0x7f24d8165be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36512/0 (socket says 172.21.15.135:36512) 2024-04-17T06:09:39.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f24d77fe640 1 -- 172.21.15.135:0/836789061 learned_addr learned my addr 172.21.15.135:0/836789061 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e759e640 1 -- 172.21.15.135:0/1670321921 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f16e00e26a0 con 0x7f16e0053c00 2024-04-17T06:09:39.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e759e640 1 -- 172.21.15.135:0/1670321921 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f16e00552c0 con 0x7f16e0054b50 2024-04-17T06:09:39.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f5648b85640 1 -- 172.21.15.135:0/744845133 >> v2:172.21.15.190:3300/0 conn(0x7f56440b3f90 msgr2=0x7f564406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f5648b85640 1 --2- 172.21.15.135:0/744845133 >> v2:172.21.15.190:3300/0 conn(0x7f56440b3f90 0x7f564406e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f5648b85640 1 -- 172.21.15.135:0/744845133 >> v1:172.21.15.135:6789/0 conn(0x7f56440b3b30 legacy=0x7f564406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f5648b85640 1 -- 172.21.15.135:0/744845133 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564406f650 con 0x7f56440b4440 2024-04-17T06:09:39.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e559a640 1 -- 172.21.15.135:0/1670321921 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2546004390 0 0) 0x7f16e0054fe0 con 0x7f16e00e12b0 2024-04-17T06:09:39.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7fafad3d2640 1 --2- 172.21.15.135:0/1600685081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa0006810 0x7fafa0006ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000b60 tx=0x7fafa80a72e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfc2d5aa41ab6863 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f24d5ffb640 1 -- 172.21.15.135:0/836789061 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2043503636 0 0) 0x7f24d806c440 con 0x7f24d8067720 2024-04-17T06:09:39.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f24d5ffb640 1 -- 172.21.15.135:0/836789061 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24c0003520 con 0x7f24d8067720 2024-04-17T06:09:39.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.587+0000 7f642f1d4640 1 -- start start 2024-04-17T06:09:39.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f2d1d257640 1 --2- 172.21.15.135:0/834941097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d18070850 0x7f2d1806c3a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009870 tx=0x7f2d0802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8827e9abc81f28a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fd758b3d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd75415aed0 0x7fd75406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fafa77fe640 1 -- 172.21.15.135:0/1600685081 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa80b4070 con 0x7fafa0006810 2024-04-17T06:09:39.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642f1d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f6420053520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fafa77fe640 1 -- 172.21.15.135:0/1600685081 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa80af640 con 0x7fafa0006810 2024-04-17T06:09:39.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fafa77fe640 1 -- 172.21.15.135:0/1600685081 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa80afab0 con 0x7fafa0006810 2024-04-17T06:09:39.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f24d67fc640 1 -- 172.21.15.135:0/836789061 >> v2:172.21.15.190:3300/0 conn(0x7f24d815bdd0 msgr2=0x7f24d8165be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f24d67fc640 1 --2- 172.21.15.135:0/836789061 >> v2:172.21.15.190:3300/0 conn(0x7f24d815bdd0 0x7f24d8165be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f24d67fc640 1 -- 172.21.15.135:0/836789061 >> v1:172.21.15.190:6789/0 conn(0x7f24d8067720 legacy=0x7f24d80b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f24d67fc640 1 -- 172.21.15.135:0/836789061 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d806ca00 con 0x7f24d8067b00 2024-04-17T06:09:39.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f5648b85640 1 --2- 172.21.15.135:0/744845133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56440b4440 0x7f56440b4820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5644071d10 tx=0x7f563002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf64963dcc12a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fd758b3d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd75406ed00 0x7fd75415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642f1d4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6420054b80 0x7f6420053a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642cf70640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f6420053520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642cf70640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f6420053520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:49922/0 (socket says 172.21.15.135:49922) 2024-04-17T06:09:39.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fd75259c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd75415aed0 0x7fd75406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fd75259c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd75415aed0 0x7fd75406e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36520/0 (socket says 172.21.15.135:36520) 2024-04-17T06:09:39.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fd75259c640 1 -- 172.21.15.135:0/4053992906 learned_addr learned my addr 172.21.15.135:0/4053992906 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f6427fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6420054b80 0x7f6420053a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f6427fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6420054b80 0x7f6420053a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48790/0 (socket says 172.21.15.135:48790) 2024-04-17T06:09:39.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fd758b3d640 1 -- 172.21.15.135:0/4053992906 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd75406f1b0 con 0x7fd75406fe40 2024-04-17T06:09:39.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fd758b3d640 1 -- 172.21.15.135:0/4053992906 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd7540b3a70 con 0x7fd75406ed00 2024-04-17T06:09:39.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7fd758b3d640 1 -- 172.21.15.135:0/4053992906 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd75406f480 con 0x7fd75415aed0 2024-04-17T06:09:39.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642f1d4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64200e4d60 0x7f64200e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642cf70640 1 -- 172.21.15.135:0/1974029735 learned_addr learned my addr 172.21.15.135:0/1974029735 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f5649386640 1 --2- 172.21.15.135:0/744845133 >> v2:172.21.15.190:3300/0 conn(0x7f56440b3f90 0x7f564406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:39.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f12a9dff640 1 Processor -- start 2024-04-17T06:09:39.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f56437fe640 1 -- 172.21.15.135:0/744845133 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630008df0 con 0x7f56440b4440 2024-04-17T06:09:39.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f56437fe640 1 -- 172.21.15.135:0/744845133 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5630037070 con 0x7f56440b4440 2024-04-17T06:09:39.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7fd74bfff640 1 --2- 172.21.15.135:0/4053992906 >> v2:172.21.15.135:3300/0 conn(0x7fd75406ed00 0x7fd75415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f56437fe640 1 -- 172.21.15.135:0/744845133 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630008df0 con 0x7f56440b4440 2024-04-17T06:09:39.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e559a640 1 -- 172.21.15.135:0/1670321921 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16d0003520 con 0x7f16e00e12b0 2024-04-17T06:09:39.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e5d9b640 1 -- 172.21.15.135:0/1670321921 >> v2:172.21.15.135:3300/0 conn(0x7f16e0053c00 msgr2=0x7f16e0052030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e5d9b640 1 --2- 172.21.15.135:0/1670321921 >> v2:172.21.15.135:3300/0 conn(0x7f16e0053c00 0x7f16e0052030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e5d9b640 1 -- 172.21.15.135:0/1670321921 >> v1:172.21.15.190:6789/0 conn(0x7f16e00e12b0 legacy=0x7f16e00534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.588+0000 7f16e5d9b640 1 -- 172.21.15.135:0/1670321921 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e00555a0 con 0x7f16e0054b50 2024-04-17T06:09:39.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7fd75259c640 1 -- 172.21.15.135:0/4053992906 >> v2:172.21.15.135:3300/0 conn(0x7fd75406ed00 msgr2=0x7fd75415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7fd75259c640 1 --2- 172.21.15.135:0/4053992906 >> v2:172.21.15.135:3300/0 conn(0x7fd75406ed00 0x7fd75415b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7fd75259c640 1 -- 172.21.15.135:0/4053992906 >> v1:172.21.15.190:6789/0 conn(0x7fd75406fe40 legacy=0x7fd7540b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7fd75259c640 1 -- 172.21.15.135:0/4053992906 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7540b3420 con 0x7fd75415aed0 2024-04-17T06:09:39.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f12a9dff640 1 -- start start 2024-04-17T06:09:39.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642f1d4640 1 -- 172.21.15.135:0/1974029735 --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6420055010 con 0x7f6420054b80 2024-04-17T06:09:39.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642f1d4640 1 -- 172.21.15.135:0/1974029735 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f64200552f0 con 0x7f64200e4d60 2024-04-17T06:09:39.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f642f1d4640 1 -- 172.21.15.135:0/1974029735 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f64200555d0 con 0x7f64200e12e0 2024-04-17T06:09:39.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f2d177fe640 1 -- 172.21.15.135:0/834941097 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08035020 con 0x7f2d18070850 2024-04-17T06:09:39.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f16e5d9b640 1 --2- 172.21.15.135:0/1670321921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0054b50 0x7f16e0051af0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0097a0 tx=0x7f16dc02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eecd69c62484ac52 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f24d67fc640 1 --2- 172.21.15.135:0/836789061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d8067b00 0x7f24d815b890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f24d80b29e0 tx=0x7f24cc02fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40cd869de7836a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f642d771640 1 --2- 172.21.15.135:0/1974029735 >> v2:172.21.15.190:3300/0 conn(0x7f64200e4d60 0x7f64200e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.589+0000 7f2d177fe640 1 -- 172.21.15.135:0/834941097 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08031890 con 0x7f2d18070850 2024-04-17T06:09:39.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f2d1da58640 1 -- 172.21.15.135:0/834941097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d18070850 msgr2=0x7f2d1806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f2d1da58640 1 --2- 172.21.15.135:0/834941097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d18070850 0x7f2d1806c3a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009870 tx=0x7f2d0802b930 comp rx=0 tx=0).stop 2024-04-17T06:09:39.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f2d1da58640 1 -- 172.21.15.135:0/834941097 shutdown_connections 2024-04-17T06:09:39.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f2d1da58640 1 --2- 172.21.15.135:0/834941097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d18070850 0x7f2d1806c3a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f2d1da58640 1 -- 172.21.15.135:0/834941097 >> 172.21.15.135:0/834941097 conn(0x7f2d18073360 msgr2=0x7f2d1809a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f642cf70640 1 -- 172.21.15.135:0/1974029735 >> v2:172.21.15.190:3300/0 conn(0x7f64200e4d60 msgr2=0x7f64200e16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f564ade9640 1 -- 172.21.15.135:0/744845133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56440b4440 msgr2=0x7f56440b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f24d5ffb640 1 -- 172.21.15.135:0/836789061 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc030880 con 0x7f24d8067b00 2024-04-17T06:09:39.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f24d5ffb640 1 -- 172.21.15.135:0/836789061 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24cc030a20 con 0x7f24d8067b00 2024-04-17T06:09:39.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f24d5ffb640 1 -- 172.21.15.135:0/836789061 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc030df0 con 0x7f24d8067b00 2024-04-17T06:09:39.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f564ade9640 1 --2- 172.21.15.135:0/744845133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56440b4440 0x7f56440b4820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5644071d10 tx=0x7f563002c900 comp rx=0 tx=0).stop 2024-04-17T06:09:39.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f642cf70640 1 --2- 172.21.15.135:0/1974029735 >> v2:172.21.15.190:3300/0 conn(0x7f64200e4d60 0x7f64200e16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.611 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f642cf70640 1 -- 172.21.15.135:0/1974029735 >> v2:172.21.15.135:3300/0 conn(0x7f6420054b80 msgr2=0x7f6420053a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:39.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f564ade9640 1 -- 172.21.15.135:0/744845133 shutdown_connections 2024-04-17T06:09:39.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f642cf70640 1 --2- 172.21.15.135:0/1974029735 >> v2:172.21.15.135:3300/0 conn(0x7f6420054b80 0x7f6420053a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.590+0000 7f642cf70640 1 -- 172.21.15.135:0/1974029735 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64200558b0 con 0x7f64200e12e0 2024-04-17T06:09:39.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f642d771640 1 --2- 172.21.15.135:0/1974029735 >> v2:172.21.15.190:3300/0 conn(0x7f64200e4d60 0x7f64200e16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:39.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f564ade9640 1 --2- 172.21.15.135:0/744845133 >> v2:172.21.15.190:3300/0 conn(0x7f56440b3f90 0x7f564406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f564ade9640 1 --2- 172.21.15.135:0/744845133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56440b4440 0x7f56440b4820 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f12a9dff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f12a415d2f0 0x7f12a40b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f564ade9640 1 -- 172.21.15.135:0/744845133 >> 172.21.15.135:0/744845133 conn(0x7f56440d0520 msgr2=0x7f56440d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f2d1da58640 1 -- 172.21.15.135:0/834941097 shutdown_connections 2024-04-17T06:09:39.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f564ade9640 1 -- 172.21.15.135:0/744845133 shutdown_connections 2024-04-17T06:09:39.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f12a9dff640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a415d6d0 con 0x7f12a406e070 2024-04-17T06:09:39.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f12a9dff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a4166ca0 con 0x7f12a4166820 2024-04-17T06:09:39.614 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f12a9dff640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f12a41634d0 con 0x7f12a415d2f0 2024-04-17T06:09:39.615 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f564ade9640 1 -- 172.21.15.135:0/744845133 wait complete. 2024-04-17T06:09:39.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f6427fff640 1 --2- 172.21.15.135:0/1974029735 >> v2:172.21.15.135:3300/0 conn(0x7f6420054b80 0x7f6420053a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:39.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7fafaee35640 1 -- 172.21.15.135:0/1600685081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa0006810 msgr2=0x7fafa0006ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f16e559a640 1 -- 172.21.15.135:0/1670321921 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc035020 con 0x7f16e0054b50 2024-04-17T06:09:39.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7fafaee35640 1 --2- 172.21.15.135:0/1600685081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa0006810 0x7fafa0006ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000b60 tx=0x7fafa80a72e0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f12a37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f12a415d2f0 0x7f12a40b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f12a37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f12a415d2f0 0x7f12a40b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48800/0 (socket says 172.21.15.135:48800) 2024-04-17T06:09:39.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f12a37fe640 1 -- 172.21.15.135:0/2047599059 learned_addr learned my addr 172.21.15.135:0/2047599059 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f16e559a640 1 -- 172.21.15.135:0/1670321921 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc031810 con 0x7f16e0054b50 2024-04-17T06:09:39.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f16e559a640 1 -- 172.21.15.135:0/1670321921 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc031be0 con 0x7f16e0054b50 2024-04-17T06:09:39.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7fafaee35640 1 -- 172.21.15.135:0/1600685081 shutdown_connections 2024-04-17T06:09:39.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7fafaee35640 1 --2- 172.21.15.135:0/1600685081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa0006810 0x7fafa0006ca0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7fafaee35640 1 --2- 172.21.15.135:0/1600685081 >> v2:172.21.15.190:3300/0 conn(0x7fafa0005e90 0x7fafa00062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7fafaee35640 1 -- 172.21.15.135:0/1600685081 >> 172.21.15.135:0/1600685081 conn(0x7fafa0009fb0 msgr2=0x7fafa000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7fd75259c640 1 --2- 172.21.15.135:0/4053992906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75415aed0 0x7fd75406e7c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd754070ea0 tx=0x7fd74402ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e1312c1944837f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f24dd396640 1 -- 172.21.15.135:0/836789061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d8067b00 msgr2=0x7f24d815b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f24dd396640 1 --2- 172.21.15.135:0/836789061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d8067b00 0x7f24d815b890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f24d80b29e0 tx=0x7f24cc02fce0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7fafaee35640 1 -- 172.21.15.135:0/1600685081 shutdown_connections 2024-04-17T06:09:39.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f24dd396640 1 -- 172.21.15.135:0/836789061 shutdown_connections 2024-04-17T06:09:39.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f24dd396640 1 --2- 172.21.15.135:0/836789061 >> v2:172.21.15.190:3300/0 conn(0x7f24d815bdd0 0x7f24d8165be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f24dd396640 1 --2- 172.21.15.135:0/836789061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d8067b00 0x7f24d815b890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f24dd396640 1 -- 172.21.15.135:0/836789061 >> 172.21.15.135:0/836789061 conn(0x7f24d80cd7e0 msgr2=0x7f24d80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7fafaee35640 1 -- 172.21.15.135:0/1600685081 wait complete. 2024-04-17T06:09:39.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.591+0000 7f2d1da58640 1 -- 172.21.15.135:0/834941097 wait complete. 2024-04-17T06:09:39.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7fd751d9b640 1 -- 172.21.15.135:0/4053992906 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744034070 con 0x7fd75415aed0 2024-04-17T06:09:39.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f12a27fc640 1 -- 172.21.15.135:0/2047599059 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3869534366 0 0) 0x7f12a4166ca0 con 0x7f12a4166820 2024-04-17T06:09:39.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f12a27fc640 1 -- 172.21.15.135:0/2047599059 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1294003520 con 0x7f12a4166820 2024-04-17T06:09:39.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f12a27fc640 1 -- 172.21.15.135:0/2047599059 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1145437274 0 0) 0x7f12a415d6d0 con 0x7f12a406e070 2024-04-17T06:09:39.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f12a37fe640 1 -- 172.21.15.135:0/2047599059 >> v1:172.21.15.190:6789/0 conn(0x7f12a4166820 legacy=0x7f12a4163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f12a37fe640 1 -- 172.21.15.135:0/2047599059 >> v1:172.21.15.135:6789/0 conn(0x7f12a406e070 legacy=0x7f12a40b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f12a37fe640 1 -- 172.21.15.135:0/2047599059 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a4166f70 con 0x7f12a415d2f0 2024-04-17T06:09:39.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7fd751d9b640 1 -- 172.21.15.135:0/4053992906 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd744008df0 con 0x7fd75415aed0 2024-04-17T06:09:39.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7fd751d9b640 1 -- 172.21.15.135:0/4053992906 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74400a2f0 con 0x7fd75415aed0 2024-04-17T06:09:39.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f24dd396640 1 -- 172.21.15.135:0/836789061 shutdown_connections 2024-04-17T06:09:39.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f642cf70640 1 --2- 172.21.15.135:0/1974029735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f6420053520 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f642000c0f0 tx=0x7f641c02b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15f411543da37f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f16e759e640 1 -- 172.21.15.135:0/1670321921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0054b50 msgr2=0x7f16e0051af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f16e759e640 1 --2- 172.21.15.135:0/1670321921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0054b50 0x7f16e0051af0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0097a0 tx=0x7f16dc02b8b0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f16e759e640 1 -- 172.21.15.135:0/1670321921 shutdown_connections 2024-04-17T06:09:39.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f16e759e640 1 --2- 172.21.15.135:0/1670321921 >> v2:172.21.15.135:3300/0 conn(0x7f16e0053c00 0x7f16e0052030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f16e759e640 1 --2- 172.21.15.135:0/1670321921 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0054b50 0x7f16e0051af0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0097a0 tx=0x7f16dc02b8b0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f64277fe640 1 -- 172.21.15.135:0/1974029735 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c035020 con 0x7f64200e12e0 2024-04-17T06:09:39.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f64277fe640 1 -- 172.21.15.135:0/1974029735 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641c031880 con 0x7f64200e12e0 2024-04-17T06:09:39.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.592+0000 7f16e759e640 1 -- 172.21.15.135:0/1670321921 >> 172.21.15.135:0/1670321921 conn(0x7f16e0020040 msgr2=0x7f16e0056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f64277fe640 1 -- 172.21.15.135:0/1974029735 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c031c50 con 0x7f64200e12e0 2024-04-17T06:09:39.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f2d1da58640 1 Processor -- start 2024-04-17T06:09:39.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f564ade9640 1 Processor -- start 2024-04-17T06:09:39.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f24dd396640 1 -- 172.21.15.135:0/836789061 wait complete. 2024-04-17T06:09:39.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f564ade9640 1 -- start start 2024-04-17T06:09:39.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7fd758b3d640 1 -- 172.21.15.135:0/4053992906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75415aed0 msgr2=0x7fd75406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f12a37fe640 1 --2- 172.21.15.135:0/2047599059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a415d2f0 0x7f12a40b3540 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f129c009e50 tx=0x7f129c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b87afc16433f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7fd758b3d640 1 --2- 172.21.15.135:0/4053992906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75415aed0 0x7fd75406e7c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd754070ea0 tx=0x7fd74402ca30 comp rx=0 tx=0).stop 2024-04-17T06:09:39.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7fafaee35640 1 Processor -- start 2024-04-17T06:09:39.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.593+0000 7f2d1da58640 1 -- start start 2024-04-17T06:09:39.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7fafaee35640 1 -- start start 2024-04-17T06:09:39.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7fd758b3d640 1 -- 172.21.15.135:0/4053992906 shutdown_connections 2024-04-17T06:09:39.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7fd758b3d640 1 --2- 172.21.15.135:0/4053992906 >> v2:172.21.15.135:3300/0 conn(0x7fd75406ed00 0x7fd75415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f12a27fc640 1 -- 172.21.15.135:0/2047599059 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c02cb10 con 0x7f12a415d2f0 2024-04-17T06:09:39.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7fd758b3d640 1 --2- 172.21.15.135:0/4053992906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75415aed0 0x7fd75406e7c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7fd758b3d640 1 -- 172.21.15.135:0/4053992906 >> 172.21.15.135:0/4053992906 conn(0x7fd7540b1f80 msgr2=0x7fd7540b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 -- 172.21.15.135:0/1974029735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 msgr2=0x7f6420053520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 --2- 172.21.15.135:0/1974029735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f6420053520 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f642000c0f0 tx=0x7f641c02b920 comp rx=0 tx=0).stop 2024-04-17T06:09:39.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f12a27fc640 1 -- 172.21.15.135:0/2047599059 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c02ccb0 con 0x7f12a415d2f0 2024-04-17T06:09:39.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f16e759e640 1 -- 172.21.15.135:0/1670321921 shutdown_connections 2024-04-17T06:09:39.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7fd758b3d640 1 -- 172.21.15.135:0/4053992906 shutdown_connections 2024-04-17T06:09:39.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 -- 172.21.15.135:0/1974029735 shutdown_connections 2024-04-17T06:09:39.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 --2- 172.21.15.135:0/1974029735 >> v2:172.21.15.190:3300/0 conn(0x7f64200e4d60 0x7f64200e16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 --2- 172.21.15.135:0/1974029735 >> v2:172.21.15.135:3300/0 conn(0x7f6420054b80 0x7f6420053a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 --2- 172.21.15.135:0/1974029735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f6420053520 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 -- 172.21.15.135:0/1974029735 >> 172.21.15.135:0/1974029735 conn(0x7f6420020070 msgr2=0x7f6420056da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f12a27fc640 1 -- 172.21.15.135:0/2047599059 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c02ce60 con 0x7f12a415d2f0 2024-04-17T06:09:39.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 -- 172.21.15.135:0/1974029735 shutdown_connections 2024-04-17T06:09:39.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f642f1d4640 1 -- 172.21.15.135:0/1974029735 wait complete. 2024-04-17T06:09:39.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f2d1da58640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d18070850 0x7f2d1815f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f2d1da58640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d180b6ee0 0x7f2d180ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f564ade9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56440b3b30 0x7f564414b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f12a9dff640 1 -- 172.21.15.135:0/2047599059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a415d2f0 msgr2=0x7f12a40b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f12a9dff640 1 --2- 172.21.15.135:0/2047599059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a415d2f0 0x7f12a40b3540 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f129c009e50 tx=0x7f129c02bf70 comp rx=0 tx=0).stop 2024-04-17T06:09:39.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafaee35640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa0005e90 0x7fafa0049ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafaee35640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa0006810 0x7fafa0047450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f564ade9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56440b3f90 0x7f56440ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafaee35640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa00ead00 0x7fafa004e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafaee35640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fafa00547c0 con 0x7fafa0006810 2024-04-17T06:09:39.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafaee35640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fafa00501a0 con 0x7fafa0005e90 2024-04-17T06:09:39.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafaee35640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fafa0054940 con 0x7fafa00ead00 2024-04-17T06:09:39.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f2d1da58640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d181609b0 0x7f2d18165450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f564ade9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b4440 0x7f56440aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f2d1da58640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18163df0 con 0x7f2d181609b0 2024-04-17T06:09:39.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f2d1da58640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18163c70 con 0x7f2d18070850 2024-04-17T06:09:39.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafa7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa0006810 0x7fafa0047450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafa7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa0006810 0x7fafa0047450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48808/0 (socket says 172.21.15.135:48808) 2024-04-17T06:09:39.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafa7fff640 1 -- 172.21.15.135:0/3925934958 learned_addr learned my addr 172.21.15.135:0/3925934958 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f5643fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56440b3f90 0x7f56440ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f5643fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56440b3f90 0x7f56440ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:49930/0 (socket says 172.21.15.135:49930) 2024-04-17T06:09:39.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f24dd396640 1 Processor -- start 2024-04-17T06:09:39.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f5643fff640 1 -- 172.21.15.135:0/3179644586 learned_addr learned my addr 172.21.15.135:0/3179644586 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f5649386640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b4440 0x7f56440aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.639 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f24dd396640 1 -- start start 2024-04-17T06:09:39.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f564ade9640 1 -- 172.21.15.135:0/3179644586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f564415ae30 con 0x7f56440b3b30 2024-04-17T06:09:39.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f2d1da58640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f2d18163f70 con 0x7f2d180b6ee0 2024-04-17T06:09:39.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f564ade9640 1 -- 172.21.15.135:0/3179644586 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f564406fc10 con 0x7f56440b4440 2024-04-17T06:09:39.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f12a9dff640 1 -- 172.21.15.135:0/2047599059 shutdown_connections 2024-04-17T06:09:39.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f12a9dff640 1 --2- 172.21.15.135:0/2047599059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a415d2f0 0x7f12a40b3540 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f564ade9640 1 -- 172.21.15.135:0/3179644586 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f564415afb0 con 0x7f56440b3f90 2024-04-17T06:09:39.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f12a9dff640 1 -- 172.21.15.135:0/2047599059 >> 172.21.15.135:0/2047599059 conn(0x7f12a4067d50 msgr2=0x7f12a40d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:39.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f12a9dff640 1 -- 172.21.15.135:0/2047599059 shutdown_connections 2024-04-17T06:09:39.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafa7fff640 1 -- 172.21.15.135:0/3925934958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa00ead00 msgr2=0x7fafa004e050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:09:39.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafa7fff640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa00ead00 0x7fafa004e050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafa7fff640 1 -- 172.21.15.135:0/3925934958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa0005e90 msgr2=0x7fafa0049ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:09:39.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafa7fff640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa0005e90 0x7fafa0049ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7fafa7fff640 1 -- 172.21.15.135:0/3925934958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa0047990 con 0x7fafa0006810 2024-04-17T06:09:39.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f2d1d257640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d181609b0 0x7f2d18165450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7fafa7fff640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa0006810 0x7fafa0047450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b450 tx=0x7faf9400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.595+0000 7f12a9dff640 1 -- 172.21.15.135:0/2047599059 wait complete. 2024-04-17T06:09:39.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f24dd396640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d8067720 0x7f24d81e4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f24dd396640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d8067b00 0x7f24d806ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24d6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d8067720 0x7f24d81e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24d6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d8067720 0x7f24d81e4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36572/0 (socket says 172.21.15.135:36572) 2024-04-17T06:09:39.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24d6ffd640 1 -- 172.21.15.135:0/3773015081 learned_addr learned my addr 172.21.15.135:0/3773015081 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24d67fc640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d8067b00 0x7f24d806ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24dd396640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d815bdd0 0x7f24d814a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7fafa57fa640 1 -- 172.21.15.135:0/3925934958 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94015020 con 0x7fafa0006810 2024-04-17T06:09:39.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa004a0f0 con 0x7fafa0006810 2024-04-17T06:09:39.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7fafa57fa640 1 -- 172.21.15.135:0/3925934958 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf94011890 con 0x7fafa0006810 2024-04-17T06:09:39.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa00353f0 con 0x7fafa0006810 2024-04-17T06:09:39.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.594+0000 7f16e759e640 1 -- 172.21.15.135:0/1670321921 wait complete. 2024-04-17T06:09:39.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f16e759e640 1 Processor -- start 2024-04-17T06:09:39.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f5643fff640 1 -- 172.21.15.135:0/3179644586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b4440 msgr2=0x7f56440aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f5643fff640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b4440 0x7f56440aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f5643fff640 1 -- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56440b3b30 msgr2=0x7f564414b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f5643fff640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56440b3b30 0x7f564414b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f5643fff640 1 -- 172.21.15.135:0/3179644586 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56440ac120 con 0x7f56440b3f90 2024-04-17T06:09:39.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f5643fff640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56440b3f90 0x7f56440ae540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f563800b3e0 tx=0x7f563800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7fd758b3d640 1 -- 172.21.15.135:0/4053992906 wait complete. 2024-04-17T06:09:39.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f642f1d4640 1 Processor -- start 2024-04-17T06:09:39.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f642f1d4640 1 -- start start 2024-04-17T06:09:39.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f2d1d257640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d181609b0 0x7f2d18165450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48810/0 (socket says 172.21.15.135:48810) 2024-04-17T06:09:39.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f2d1d257640 1 -- 172.21.15.135:0/2220680586 learned_addr learned my addr 172.21.15.135:0/2220680586 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f2d1ca56640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d18070850 0x7f2d1815f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24d77fe640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d815bdd0 0x7f24d814a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d80b3ea0 con 0x7f24d815bdd0 2024-04-17T06:09:39.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f16e759e640 1 -- start start 2024-04-17T06:09:39.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f16e759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0053c00 0x7f16e00e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f24d80b4f00 con 0x7f24d8067720 2024-04-17T06:09:39.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f16e759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16e00e12b0 0x7f16e00df650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa0005390 con 0x7fafa0006810 2024-04-17T06:09:39.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.596+0000 7f2d17fff640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d180b6ee0 0x7f2d180ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7fafa57fa640 1 -- 172.21.15.135:0/3925934958 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94011c40 con 0x7fafa0006810 2024-04-17T06:09:39.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f16e759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16e00dfb90 0x7f16e00de180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f16e759e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0057820 con 0x7f16e00dfb90 2024-04-17T06:09:39.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f16e759e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16e00576a0 con 0x7f16e00e12b0 2024-04-17T06:09:39.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f16e759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f16e00e4d30 con 0x7f16e0053c00 2024-04-17T06:09:39.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f16e5d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16e00e12b0 0x7f16e00df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f56417fa640 1 -- 172.21.15.135:0/3179644586 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638015020 con 0x7f56440b3f90 2024-04-17T06:09:39.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f56417fa640 1 -- 172.21.15.135:0/3179644586 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5638011850 con 0x7f56440b3f90 2024-04-17T06:09:39.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f56417fa640 1 -- 172.21.15.135:0/3179644586 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638011c40 con 0x7f56440b3f90 2024-04-17T06:09:39.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f564ade9640 1 -- 172.21.15.135:0/3179644586 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56440ac400 con 0x7f56440b3f90 2024-04-17T06:09:39.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f564ade9640 1 -- 172.21.15.135:0/3179644586 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56440ac9c0 con 0x7f56440b3f90 2024-04-17T06:09:39.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f642f1d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f64200501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f642f1d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64200e4d60 0x7f64200506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f12a9dff640 1 Processor -- start 2024-04-17T06:09:39.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f12a9dff640 1 -- start start 2024-04-17T06:09:39.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7fd758b3d640 1 Processor -- start 2024-04-17T06:09:39.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f6427fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64200e4d60 0x7f64200506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f6427fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64200e4d60 0x7f64200506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48848/0 (socket says 172.21.15.135:48848) 2024-04-17T06:09:39.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f6427fff640 1 -- 172.21.15.135:0/818616134 learned_addr learned my addr 172.21.15.135:0/818616134 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.597+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f24d80b4020 con 0x7f24d8067b00 2024-04-17T06:09:39.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7fd758b3d640 1 -- start start 2024-04-17T06:09:39.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f2d1d257640 1 -- 172.21.15.135:0/2220680586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d180b6ee0 msgr2=0x7f2d180ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f2d1d257640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d180b6ee0 0x7f2d180ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f2d1d257640 1 -- 172.21.15.135:0/2220680586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d18070850 msgr2=0x7f2d1815f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f2d1d257640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d18070850 0x7f2d1815f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f2d1d257640 1 -- 172.21.15.135:0/2220680586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d181448d0 con 0x7f2d181609b0 2024-04-17T06:09:39.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.598+0000 7f16e659c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0053c00 0x7f16e00e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f16e659c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0053c00 0x7f16e00e3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:49956/0 (socket says 172.21.15.135:49956) 2024-04-17T06:09:39.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f16e659c640 1 -- 172.21.15.135:0/1520756279 learned_addr learned my addr 172.21.15.135:0/1520756279 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f642f1d4640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f642004de70 0x7f642004e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f642f1d4640 1 -- 172.21.15.135:0/818616134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64200e1d80 con 0x7f64200e4d60 2024-04-17T06:09:39.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f642cf70640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f64200501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f2d1d257640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d181609b0 0x7f2d18165450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009990 tx=0x7f2d080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f16e6d9d640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16e00dfb90 0x7f16e00de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f642f1d4640 1 -- 172.21.15.135:0/818616134 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f64200e1c00 con 0x7f642004de70 2024-04-17T06:09:39.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f642f1d4640 1 -- 172.21.15.135:0/818616134 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f64200e1f00 con 0x7f64200e12e0 2024-04-17T06:09:39.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f642d771640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f642004de70 0x7f642004e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.656 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a9dff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12a406e070 0x7f12a415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a9dff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a415d2f0 0x7f12a40ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a9dff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a4166820 0x7f12a4158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a9dff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4164070 con 0x7f12a4166820 2024-04-17T06:09:39.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a9dff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4167530 con 0x7f12a415d2f0 2024-04-17T06:09:39.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a9dff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f12a41641f0 con 0x7f12a406e070 2024-04-17T06:09:39.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a4166820 0x7f12a4158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.657 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f2d157fa640 1 -- 172.21.15.135:0/2220680586 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08040070 con 0x7f2d181609b0 2024-04-17T06:09:39.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a415d2f0 0x7f12a40ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a2ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a415d2f0 0x7f12a40ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36598/0 (socket says 172.21.15.135:36598) 2024-04-17T06:09:39.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f12a2ffd640 1 -- 172.21.15.135:0/4294277384 learned_addr learned my addr 172.21.15.135:0/4294277384 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f2d157fa640 1 -- 172.21.15.135:0/2220680586 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0803a830 con 0x7f2d181609b0 2024-04-17T06:09:39.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f12a37fe640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12a406e070 0x7f12a415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f16e5d9b640 1 -- 172.21.15.135:0/1520756279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0053c00 msgr2=0x7f16e00e3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f16e5d9b640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0053c00 0x7f16e00e3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f16e5d9b640 1 -- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16e00dfb90 msgr2=0x7f16e00de180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f24d6ffd640 1 -- 172.21.15.135:0/3773015081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d8067b00 msgr2=0x7f24d806ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f24d6ffd640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d8067b00 0x7f24d806ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f24d6ffd640 1 -- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d815bdd0 msgr2=0x7f24d814a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f24d6ffd640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d815bdd0 0x7f24d814a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.659 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f24d6ffd640 1 -- 172.21.15.135:0/3773015081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d806f230 con 0x7f24d8067720 2024-04-17T06:09:39.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f16e5d9b640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16e00dfb90 0x7f16e00de180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f16e5d9b640 1 -- 172.21.15.135:0/1520756279 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e0180f90 con 0x7f16e00e12b0 2024-04-17T06:09:39.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.599+0000 7f564ade9640 1 -- 172.21.15.135:0/3179644586 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5644074940 con 0x7f56440b3f90 2024-04-17T06:09:39.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f16e6d9d640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16e00dfb90 0x7f16e00de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:39.660 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f56417fa640 1 -- 172.21.15.135:0/3179644586 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5638018020 con 0x7f56440b3f90 2024-04-17T06:09:39.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7fd758b3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75406ed00 0x7fd75414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f12a37fe640 1 -- 172.21.15.135:0/4294277384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a415d2f0 msgr2=0x7f12a40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f12a37fe640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a415d2f0 0x7f12a40ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f12a37fe640 1 -- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a4166820 msgr2=0x7f12a4158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f12a37fe640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a4166820 0x7f12a4158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f12a37fe640 1 -- 172.21.15.135:0/4294277384 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a4160df0 con 0x7f12a406e070 2024-04-17T06:09:39.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f642d771640 1 -- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 msgr2=0x7f64200501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f642d771640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f64200501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f642d771640 1 -- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64200e4d60 msgr2=0x7f64200506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f642d771640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64200e4d60 0x7f64200506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f642d771640 1 -- 172.21.15.135:0/818616134 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642004a8d0 con 0x7f642004de70 2024-04-17T06:09:39.662 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f24d6ffd640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d8067720 0x7f24d81e4410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f24c800c430 tx=0x7f24c800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7fd758b3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd75406fe40 0x7fd7540ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f2d1da58640 1 -- 172.21.15.135:0/2220680586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d18144bb0 con 0x7f2d181609b0 2024-04-17T06:09:39.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f16e5d9b640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16e00e12b0 0x7f16e00df650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16e0055440 tx=0x7f16dc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7fd758b3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd75415aed0 0x7fd75414a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f642d771640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f642004de70 0x7f642004e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6428050fc0 tx=0x7f6428068df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f2d1da58640 1 -- 172.21.15.135:0/2220680586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d18145170 con 0x7f2d181609b0 2024-04-17T06:09:39.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f2d157fa640 1 -- 172.21.15.135:0/2220680586 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08040070 con 0x7f2d181609b0 2024-04-17T06:09:39.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.600+0000 7f12a37fe640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12a406e070 0x7f12a415fcc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f129c032000 tx=0x7f129c004a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7fd75259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75406ed00 0x7fd75414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7fafa57fa640 1 -- 172.21.15.135:0/3925934958 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf94018020 con 0x7fafa0006810 2024-04-17T06:09:39.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f24bf7fe640 1 -- 172.21.15.135:0/3773015081 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8017070 con 0x7f24d8067720 2024-04-17T06:09:39.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f24bf7fe640 1 -- 172.21.15.135:0/3773015081 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c80035f0 con 0x7f24d8067720 2024-04-17T06:09:39.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d806f500 con 0x7f24d8067720 2024-04-17T06:09:39.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f24bf7fe640 1 -- 172.21.15.135:0/3773015081 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80039e0 con 0x7f24d8067720 2024-04-17T06:09:39.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d806fac0 con 0x7f24d8067720 2024-04-17T06:09:39.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7fafa57fa640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8c06c8f0 0x7faf8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7fafa57fa640 1 -- 172.21.15.135:0/3925934958 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faf9408bf20 con 0x7fafa0006810 2024-04-17T06:09:39.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7fafacbd1640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8c06c8f0 0x7faf8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7fafacbd1640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8c06c8f0 0x7faf8c06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faf9c007fd0 tx=0x7faf9c0079a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f16d6ffd640 1 -- 172.21.15.135:0/1520756279 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc035040 con 0x7f16e00e12b0 2024-04-17T06:09:39.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f16d6ffd640 1 -- 172.21.15.135:0/1520756279 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc03aa70 con 0x7f16e00e12b0 2024-04-17T06:09:39.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f16d6ffd640 1 -- 172.21.15.135:0/1520756279 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc03ae40 con 0x7f16e00e12b0 2024-04-17T06:09:39.666 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e0181270 con 0x7f16e00e12b0 2024-04-17T06:09:39.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e00e43e0 con 0x7f16e00e12b0 2024-04-17T06:09:39.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f56417fa640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f562806c9c0 0x7f562806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f56417fa640 1 -- 172.21.15.135:0/3179644586 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f563808bd80 con 0x7f56440b3f90 2024-04-17T06:09:39.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f5648b85640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f562806c9c0 0x7f562806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7fd75259c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75406ed00 0x7fd75414f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36600/0 (socket says 172.21.15.135:36600) 2024-04-17T06:09:39.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7fd75259c640 1 -- 172.21.15.135:0/2486168044 learned_addr learned my addr 172.21.15.135:0/2486168044 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:39.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd75415b970 con 0x7fd75415aed0 2024-04-17T06:09:39.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd75415b7f0 con 0x7fd75406ed00 2024-04-17T06:09:39.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd75415baf0 con 0x7fd75406fe40 2024-04-17T06:09:39.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7fd74bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd75406fe40 0x7fd7540ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7fd752d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd75415aed0 0x7fd75414a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f64257fa640 1 -- 172.21.15.135:0/818616134 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428075070 con 0x7f642004de70 2024-04-17T06:09:39.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f642f1d4640 1 -- 172.21.15.135:0/818616134 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64200d61d0 con 0x7f642004de70 2024-04-17T06:09:39.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f64257fa640 1 -- 172.21.15.135:0/818616134 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6428075210 con 0x7f642004de70 2024-04-17T06:09:39.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.601+0000 7f642f1d4640 1 -- 172.21.15.135:0/818616134 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64200d6790 con 0x7f642004de70 2024-04-17T06:09:39.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.602+0000 7f64257fa640 1 -- 172.21.15.135:0/818616134 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428075070 con 0x7f642004de70 2024-04-17T06:09:39.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7f2d157fa640 1 -- 172.21.15.135:0/2220680586 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d0804b060 con 0x7f2d181609b0 2024-04-17T06:09:39.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7f5648b85640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f562806c9c0 0x7f562806ee80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5630036000 tx=0x7f5630031000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7f2d157fa640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2d0406caf0 0x7f2d0406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7f6412ffd640 1 -- 172.21.15.135:0/818616134 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6420005810 con 0x7f642004de70 2024-04-17T06:09:39.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7f2cfeffd640 1 -- 172.21.15.135:0/2220680586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d18068a20 con 0x7f2d181609b0 2024-04-17T06:09:39.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7fd74bfff640 1 -- 172.21.15.135:0/2486168044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75406ed00 msgr2=0x7fd75414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7f2d157fa640 1 -- 172.21.15.135:0/2220680586 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2d080adaf0 con 0x7f2d181609b0 2024-04-17T06:09:39.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7fd74bfff640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75406ed00 0x7fd75414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7fd74bfff640 1 -- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd75415aed0 msgr2=0x7fd75414a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:39.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7fd74bfff640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd75415aed0 0x7fd75414a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:39.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7f2d1ca56640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2d0406caf0 0x7f2d0406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7fd74bfff640 1 -- 172.21.15.135:0/2486168044 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75414bc30 con 0x7fd75406fe40 2024-04-17T06:09:39.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.604+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d80b4440 con 0x7f24d8067720 2024-04-17T06:09:39.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.603+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e00057e0 con 0x7f16e00e12b0 2024-04-17T06:09:39.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.604+0000 7fd75259c640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75406ed00 0x7fd75414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:39.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.604+0000 7f2d1ca56640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2d0406caf0 0x7f2d0406efb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009fd0 tx=0x7f2d1000d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.604+0000 7fd74bfff640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd75406fe40 0x7fd7540ae4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00e880 tx=0x7fd73c00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.604+0000 7f64257fa640 1 -- 172.21.15.135:0/818616134 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6428075480 con 0x7f642004de70 2024-04-17T06:09:39.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f24bf7fe640 1 -- 172.21.15.135:0/3773015081 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f24c801a070 con 0x7f24d8067720 2024-04-17T06:09:39.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.604+0000 7f16d6ffd640 1 -- 172.21.15.135:0/1520756279 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16dc038020 con 0x7f16e00e12b0 2024-04-17T06:09:39.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f16d6ffd640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f16c806c970 0x7f16c806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f16d6ffd640 1 -- 172.21.15.135:0/1520756279 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f16dc0ad7c0 con 0x7f16e00e12b0 2024-04-17T06:09:39.674 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f64257fa640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f640c06cd20 0x7f640c06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f16e659c640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f16c806c970 0x7f16c806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f64257fa640 1 -- 172.21.15.135:0/818616134 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6428074e70 con 0x7f642004de70 2024-04-17T06:09:39.675 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f642cf70640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f640c06cd20 0x7f640c06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f24bf7fe640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24c406c940 0x7f24c406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f16e659c640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f16c806c970 0x7f16c806ee30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16d80011e0 tx=0x7f16d8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.605+0000 7f24bf7fe640 1 -- 172.21.15.135:0/3773015081 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f24c808b190 con 0x7f24d8067720 2024-04-17T06:09:39.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.606+0000 7f24d67fc640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24c406c940 0x7f24c406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.606+0000 7f642cf70640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f640c06cd20 0x7f640c06f1e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f641c036000 tx=0x7f641c03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.606+0000 7f24d67fc640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24c406c940 0x7f24c406ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24cc039000 tx=0x7f24cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7fd74affd640 1 -- 172.21.15.135:0/2486168044 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c017070 con 0x7fd75406fe40 2024-04-17T06:09:39.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7f128ffff640 1 -- 172.21.15.135:0/4294277384 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c043020 con 0x7f12a406e070 2024-04-17T06:09:39.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7f128ffff640 1 -- 172.21.15.135:0/4294277384 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c03e450 con 0x7f12a406e070 2024-04-17T06:09:39.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7f128ffff640 1 -- 172.21.15.135:0/4294277384 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c02ce60 con 0x7f12a406e070 2024-04-17T06:09:39.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a4161090 con 0x7f12a406e070 2024-04-17T06:09:39.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7541f0f10 con 0x7fd75406fe40 2024-04-17T06:09:39.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7fd74affd640 1 -- 172.21.15.135:0/2486168044 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c00c940 con 0x7fd75406fe40 2024-04-17T06:09:39.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7fafa57fa640 1 -- 172.21.15.135:0/3925934958 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf9401f020 con 0x7fafa0006810 2024-04-17T06:09:39.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7f56417fa640 1 -- 172.21.15.135:0/3179644586 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f563801f020 con 0x7f56440b3f90 2024-04-17T06:09:39.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.608+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a4161650 con 0x7f12a406e070 2024-04-17T06:09:39.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.609+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75414e960 con 0x7fd75406fe40 2024-04-17T06:09:39.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.609+0000 7fd74affd640 1 -- 172.21.15.135:0/2486168044 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00cd30 con 0x7fd75406fe40 2024-04-17T06:09:39.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.610+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a404fd50 con 0x7f12a406e070 2024-04-17T06:09:39.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.610+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd754074940 con 0x7fd75406fe40 2024-04-17T06:09:39.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.611+0000 7fd74affd640 1 -- 172.21.15.135:0/2486168044 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd73c01a070 con 0x7fd75406fe40 2024-04-17T06:09:39.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.611+0000 7f128ffff640 1 -- 172.21.15.135:0/4294277384 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f129c046020 con 0x7f12a406e070 2024-04-17T06:09:39.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.611+0000 7fd74affd640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd73406eb00 0x7fd734070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.612+0000 7fd74affd640 1 -- 172.21.15.135:0/2486168044 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd73c08b690 con 0x7fd75406fe40 2024-04-17T06:09:39.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.612+0000 7fd75259c640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd73406eb00 0x7fd734070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.612+0000 7f128ffff640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f128806c8f0 0x7f128806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:39.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.612+0000 7f128ffff640 1 -- 172.21.15.135:0/4294277384 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f129c0ae6f0 con 0x7f12a406e070 2024-04-17T06:09:39.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.612+0000 7f12a2ffd640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f128806c8f0 0x7f128806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:39.682 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.612+0000 7fd75259c640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd73406eb00 0x7fd734070fc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd7440071a0 tx=0x7fd74400a000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.612+0000 7f12a2ffd640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f128806c8f0 0x7f128806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12a4072650 tx=0x7f1298009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:39.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.613+0000 7f64257fa640 1 -- 172.21.15.135:0/818616134 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64280d5380 con 0x7f642004de70 2024-04-17T06:09:39.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.614+0000 7f2d157fa640 1 -- 172.21.15.135:0/2220680586 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d0802d040 con 0x7f2d181609b0 2024-04-17T06:09:39.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.620+0000 7f128ffff640 1 -- 172.21.15.135:0/4294277384 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f129c04d030 con 0x7f12a406e070 2024-04-17T06:09:39.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.624+0000 7f16d6ffd640 1 -- 172.21.15.135:0/1520756279 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16dc07b340 con 0x7f16e00e12b0 2024-04-17T06:09:39.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.630+0000 7fd74affd640 1 -- 172.21.15.135:0/2486168044 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd75414bc30 con 0x7fd75406fe40 2024-04-17T06:09:39.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:39.631+0000 7f24bf7fe640 1 -- 172.21.15.135:0/3773015081 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24d80b4440 con 0x7f24d8067720 2024-04-17T06:09:40.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.014+0000 7f564ade9640 1 -- 172.21.15.135:0/3179644586 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f564406ebd0 con 0x7f56440b3f90 2024-04-17T06:09:40.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.014+0000 7f56417fa640 1 -- 172.21.15.135:0/3179644586 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5638059e50 con 0x7f56440b3f90 2024-04-17T06:09:40.012 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-17T06:09:40.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.018+0000 7f562effd640 1 -- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f562806c9c0 msgr2=0x7f562806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.018+0000 7f562effd640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f562806c9c0 0x7f562806ee80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5630036000 tx=0x7f5630031000 comp rx=0 tx=0).stop 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.018+0000 7f562effd640 1 -- 172.21.15.135:0/3179644586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56440b3f90 msgr2=0x7f56440ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.018+0000 7f562effd640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56440b3f90 0x7f56440ae540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f563800b3e0 tx=0x7f563800b8a0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.018+0000 7f562effd640 1 -- 172.21.15.135:0/3179644586 shutdown_connections 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.018+0000 7f562effd640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f56440b4440 0x7f56440aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.018+0000 7f562effd640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f56440b3f90 0x7f56440ae540 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.018+0000 7f562effd640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f562806c9c0 0x7f562806ee80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.019+0000 7f562effd640 1 --2- 172.21.15.135:0/3179644586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56440b3b30 0x7f564414b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.019+0000 7f562effd640 1 -- 172.21.15.135:0/3179644586 >> 172.21.15.135:0/3179644586 conn(0x7f56440d0520 msgr2=0x7f56440b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.019+0000 7f562effd640 1 -- 172.21.15.135:0/3179644586 shutdown_connections 2024-04-17T06:09:40.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.019+0000 7f562effd640 1 -- 172.21.15.135:0/3179644586 wait complete. 2024-04-17T06:09:40.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.020+0000 7f2cfeffd640 1 -- 172.21.15.135:0/2220680586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2d180b2340 con 0x7f2d181609b0 2024-04-17T06:09:40.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.020+0000 7f2d157fa640 1 -- 172.21.15.135:0/2220680586 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2d0807b470 con 0x7f2d181609b0 2024-04-17T06:09:40.018 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-17T06:09:40.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.024+0000 7f2d1da58640 1 -- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2d0406caf0 msgr2=0x7f2d0406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.024+0000 7f2d1da58640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2d0406caf0 0x7f2d0406efb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009fd0 tx=0x7f2d1000d040 comp rx=0 tx=0).stop 2024-04-17T06:09:40.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.024+0000 7f2d1da58640 1 -- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d181609b0 msgr2=0x7f2d18165450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.024+0000 7f2d1da58640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d181609b0 0x7f2d18165450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009990 tx=0x7f2d080046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.024+0000 7f2d1da58640 1 -- 172.21.15.135:0/2220680586 shutdown_connections 2024-04-17T06:09:40.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.025+0000 7f2d1da58640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f2d0406caf0 0x7f2d0406efb0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009fd0 tx=0x7f2d1000d040 comp rx=0 tx=0).stop 2024-04-17T06:09:40.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.025+0000 7f2d1da58640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d181609b0 0x7f2d18165450 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.023 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.025+0000 7f2d1da58640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2d180b6ee0 0x7f2d180ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.025+0000 7f2d1da58640 1 --2- 172.21.15.135:0/2220680586 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2d18070850 0x7f2d1815f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.025+0000 7f2d1da58640 1 -- 172.21.15.135:0/2220680586 >> 172.21.15.135:0/2220680586 conn(0x7f2d18073360 msgr2=0x7f2d18163560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:40.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.025+0000 7f2d1da58640 1 -- 172.21.15.135:0/2220680586 shutdown_connections 2024-04-17T06:09:40.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.025+0000 7f2d1da58640 1 -- 172.21.15.135:0/2220680586 wait complete. 2024-04-17T06:09:40.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.034+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f12a406f510 con 0x7f12a406e070 2024-04-17T06:09:40.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.035+0000 7f128ffff640 1 -- 172.21.15.135:0/4294277384 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f129c07c850 con 0x7f12a406e070 2024-04-17T06:09:40.033 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-17T06:09:40.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.035+0000 7f6412ffd640 1 -- 172.21.15.135:0/818616134 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6420051130 con 0x7f642004de70 2024-04-17T06:09:40.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.036+0000 7f64257fa640 1 -- 172.21.15.135:0/818616134 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6428068f20 con 0x7f642004de70 2024-04-17T06:09:40.034 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-17T06:09:40.035 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.5 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.038+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f128806c8f0 msgr2=0x7f128806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.038+0000 7f12a9dff640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f128806c8f0 0x7f128806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12a4072650 tx=0x7f1298009040 comp rx=0 tx=0).stop 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.038+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12a406e070 msgr2=0x7f12a415fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.038+0000 7f12a9dff640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12a406e070 0x7f12a415fcc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f129c032000 tx=0x7f129c004a70 comp rx=0 tx=0).stop 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.038+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 shutdown_connections 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.038+0000 7f12a9dff640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f128806c8f0 0x7f128806edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.038+0000 7f12a9dff640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12a4166820 0x7f12a4158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.038+0000 7f12a9dff640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12a415d2f0 0x7f12a40ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.039+0000 7f12a9dff640 1 --2- 172.21.15.135:0/4294277384 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12a406e070 0x7f12a415fcc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.039+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 >> 172.21.15.135:0/4294277384 conn(0x7f12a4067d50 msgr2=0x7f12a40b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.039+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 shutdown_connections 2024-04-17T06:09:40.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.039+0000 7f12a9dff640 1 -- 172.21.15.135:0/4294277384 wait complete. 2024-04-17T06:09:40.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 -- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f640c06cd20 msgr2=0x7f640c06f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f640c06cd20 0x7f640c06f1e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f641c036000 tx=0x7f641c03c040 comp rx=0 tx=0).stop 2024-04-17T06:09:40.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 -- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f642004de70 msgr2=0x7f642004e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f642004de70 0x7f642004e320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6428050fc0 tx=0x7f6428068df0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 -- 172.21.15.135:0/818616134 shutdown_connections 2024-04-17T06:09:40.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f642004de70 0x7f642004e320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f640c06cd20 0x7f640c06f1e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64200e4d60 0x7f64200506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 --2- 172.21.15.135:0/818616134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64200e12e0 0x7f64200501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.040+0000 7f6412ffd640 1 -- 172.21.15.135:0/818616134 >> 172.21.15.135:0/818616134 conn(0x7f6420020070 msgr2=0x7f6420051b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:40.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.041+0000 7f6412ffd640 1 -- 172.21.15.135:0/818616134 shutdown_connections 2024-04-17T06:09:40.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.041+0000 7f6412ffd640 1 -- 172.21.15.135:0/818616134 wait complete. 2024-04-17T06:09:40.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.7 2024-04-17T06:09:40.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.045+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f16e00dff70 con 0x7f16e00e12b0 2024-04-17T06:09:40.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.046+0000 7f16d6ffd640 1 -- 172.21.15.135:0/1520756279 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f16dc07f640 con 0x7f16e00e12b0 2024-04-17T06:09:40.044 INFO:teuthology.orchestra.run.smithi135.stdout:55834574855 2024-04-17T06:09:40.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.047+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd75415cdf0 con 0x7fd75406fe40 2024-04-17T06:09:40.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.048+0000 7fd74affd640 1 -- 172.21.15.135:0/2486168044 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd73c059810 con 0x7fd75406fe40 2024-04-17T06:09:40.046 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-17T06:09:40.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.049+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fafa0009000 con 0x7fafa0006810 2024-04-17T06:09:40.047 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.049+0000 7fafa57fa640 1 -- 172.21.15.135:0/3925934958 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf9400d040 con 0x7fafa0006810 2024-04-17T06:09:40.048 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-17T06:09:40.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f16c806c970 msgr2=0x7f16c806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f16c806c970 0x7f16c806ee30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16d80011e0 tx=0x7f16d8009040 comp rx=0 tx=0).stop 2024-04-17T06:09:40.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16e00e12b0 msgr2=0x7f16e00df650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16e00e12b0 0x7f16e00df650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f16e0055440 tx=0x7f16dc0046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 shutdown_connections 2024-04-17T06:09:40.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f16c806c970 0x7f16c806ee30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16e00dfb90 0x7f16e00de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f16e00e12b0 0x7f16e00df650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 --2- 172.21.15.135:0/1520756279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16e0053c00 0x7f16e00e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 >> 172.21.15.135:0/1520756279 conn(0x7f16e0020040 msgr2=0x7f16e0051af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:40.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 shutdown_connections 2024-04-17T06:09:40.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.050+0000 7f16e759e640 1 -- 172.21.15.135:0/1520756279 wait complete. 2024-04-17T06:09:40.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd73406eb00 msgr2=0x7fd734070fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.051 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd73406eb00 0x7fd734070fc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd7440071a0 tx=0x7fd74400a000 comp rx=0 tx=0).stop 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd75406fe40 msgr2=0x7fd7540ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd75406fe40 0x7fd7540ae4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd73c00e880 tx=0x7fd73c00ed40 comp rx=0 tx=0).stop 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 shutdown_connections 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd73406eb00 0x7fd734070fc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd75415aed0 0x7fd75414a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd75406fe40 0x7fd7540ae4b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 --2- 172.21.15.135:0/2486168044 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd75406ed00 0x7fd75414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 >> 172.21.15.135:0/2486168044 conn(0x7fd7540b1f80 msgr2=0x7fd75415d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 shutdown_connections 2024-04-17T06:09:40.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.052+0000 7fd758b3d640 1 -- 172.21.15.135:0/2486168044 wait complete. 2024-04-17T06:09:40.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8c06c8f0 msgr2=0x7faf8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8c06c8f0 0x7faf8c06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faf9c007fd0 tx=0x7faf9c0079a0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa0006810 msgr2=0x7fafa0047450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa0006810 0x7fafa0047450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf9400b450 tx=0x7faf9400b910 comp rx=0 tx=0).stop 2024-04-17T06:09:40.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 shutdown_connections 2024-04-17T06:09:40.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fafa00ead00 0x7fafa004e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faf8c06c8f0 0x7faf8c06edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafa0006810 0x7fafa0047450 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 --2- 172.21.15.135:0/3925934958 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fafa0005e90 0x7fafa0049ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 >> 172.21.15.135:0/3925934958 conn(0x7fafa0009fb0 msgr2=0x7fafa0038be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:40.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 shutdown_connections 2024-04-17T06:09:40.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.053+0000 7fafaee35640 1 -- 172.21.15.135:0/3925934958 wait complete. 2024-04-17T06:09:40.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.1 2024-04-17T06:09:40.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.3 2024-04-17T06:09:40.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2024-04-17T06:09:40.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.6 2024-04-17T06:09:40.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.4 2024-04-17T06:09:40.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.074+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f24d81594a0 con 0x7f24d8067720 2024-04-17T06:09:40.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.074+0000 7f24bf7fe640 1 -- 172.21.15.135:0/3773015081 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24c8058ca0 con 0x7f24d8067720 2024-04-17T06:09:40.072 INFO:teuthology.orchestra.run.smithi135.stdout:55834574856 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24c406c940 msgr2=0x7f24c406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24c406c940 0x7f24c406ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24cc039000 tx=0x7f24cc008040 comp rx=0 tx=0).stop 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d8067720 msgr2=0x7f24d81e4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d8067720 0x7f24d81e4410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f24c800c430 tx=0x7f24c800c8f0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 shutdown_connections 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f24c406c940 0x7f24c406ee00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d815bdd0 0x7f24d814a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d8067b00 0x7f24d806ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 --2- 172.21.15.135:0/3773015081 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d8067720 0x7f24d81e4410 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 >> 172.21.15.135:0/3773015081 conn(0x7f24d80cd7e0 msgr2=0x7f24d80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:40.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 shutdown_connections 2024-04-17T06:09:40.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:40.076+0000 7f24dd396640 1 -- 172.21.15.135:0/3773015081 wait complete. 2024-04-17T06:09:40.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.0 2024-04-17T06:09:41.036 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-17T06:09:41.043 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-17T06:09:41.057 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-17T06:09:41.057 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-17T06:09:41.066 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-17T06:09:41.066 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-17T06:09:41.069 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-17T06:09:41.086 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-17T06:09:41.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.205+0000 7fb04887b640 1 Processor -- start 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.205+0000 7fb04887b640 1 -- start start 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.206+0000 7fb04887b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb044160ba0 0x7fb04409bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.206+0000 7fb04887b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb0440c5d70 0x7fb044163ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.206+0000 7fb04887b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04409ac40 con 0x7fb04406d120 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.206+0000 7fb04887b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb0441643e0 con 0x7fb0440c5d70 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.206+0000 7fb04887b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb04409a9f0 con 0x7fb044160ba0 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.207+0000 7fb04259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb044160ba0 0x7fb04409bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.207+0000 7fb04259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb044160ba0 0x7fb04409bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:49986/0 (socket says 172.21.15.135:49986) 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.207+0000 7fb04259c640 1 -- 172.21.15.135:0/2657523846 learned_addr learned my addr 172.21.15.135:0/2657523846 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.207+0000 7fb042d9d640 1 --2- 172.21.15.135:0/2657523846 >> v2:172.21.15.190:3300/0 conn(0x7fb0440c5d70 0x7fb044163ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.207+0000 7fb041d9b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb04406d120 0x7fb0440c5660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37804/0 (socket says 172.21.15.135:37804) 2024-04-17T06:09:41.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.207+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2848371290 0 0) 0x7fb04409ac40 con 0x7fb04406d120 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb02c003520 con 0x7fb04406d120 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3443847340 0 0) 0x7fb02c003520 con 0x7fb04406d120 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb044160ba0 msgr2=0x7fb04409bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb044160ba0 0x7fb04409bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 >> v2:172.21.15.190:3300/0 conn(0x7fb0440c5d70 msgr2=0x7fb044163ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 --2- 172.21.15.135:0/2657523846 >> v2:172.21.15.190:3300/0 conn(0x7fb0440c5d70 0x7fb044163ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04409af20 con 0x7fb04406d120 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fb038001670 con 0x7fb04406d120 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 >> v1:172.21.15.135:6789/0 conn(0x7fb04406d120 legacy=0x7fb0440c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb02c003a30 0x7fb02c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb02c004800 0x7fb02c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb02c0051c0 0x7fb02c02d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441643e0 con 0x7fb02c003a30 2024-04-17T06:09:41.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb04409a9f0 con 0x7fb02c0051c0 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.208+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c02df60 con 0x7fb02c004800 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.209+0000 7fb04259c640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb02c003a30 0x7fb02c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.210+0000 7fb04259c640 1 -- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb02c004800 msgr2=0x7fb02c004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.210+0000 7fb04259c640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb02c004800 0x7fb02c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.210+0000 7fb04259c640 1 -- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb02c0051c0 msgr2=0x7fb02c02d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.209+0000 7fb042d9d640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb02c0051c0 0x7fb02c02d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.210+0000 7fb04259c640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb02c0051c0 0x7fb02c02d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.210+0000 7fb04259c640 1 -- 172.21.15.135:0/2657523846 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c003520 con 0x7fb02c003a30 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.211+0000 7fb04259c640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb02c003a30 0x7fb02c003e10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e560 tx=0x7fb02c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02aee98248425f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.211+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c009c70 con 0x7fb02c003a30 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.211+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c02cbd0 con 0x7fb02c003a30 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.211+0000 7fb04159a640 1 -- 172.21.15.135:0/2657523846 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0343f0 con 0x7fb02c003a30 2024-04-17T06:09:41.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.212+0000 7fb04887b640 1 -- 172.21.15.135:0/2657523846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb02c003a30 msgr2=0x7fb02c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.212+0000 7fb04887b640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb02c003a30 0x7fb02c003e10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00e560 tx=0x7fb02c005830 comp rx=0 tx=0).stop 2024-04-17T06:09:41.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.213+0000 7fb042d9d640 1 --2- 172.21.15.135:0/2657523846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb02c0051c0 0x7fb02c02d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:41.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.213+0000 7fb042d9d640 1 -- 172.21.15.135:0/2657523846 reap_dead start 2024-04-17T06:09:41.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.213+0000 7fb04887b640 1 -- 172.21.15.135:0/2657523846 shutdown_connections 2024-04-17T06:09:41.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.213+0000 7fb04887b640 1 -- 172.21.15.135:0/2657523846 >> 172.21.15.135:0/2657523846 conn(0x7fb0440b4180 msgr2=0x7fb0440b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.215+0000 7fb04887b640 1 -- 172.21.15.135:0/2657523846 shutdown_connections 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.215+0000 7fb04887b640 1 -- 172.21.15.135:0/2657523846 wait complete. 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.217+0000 7fb04887b640 1 Processor -- start 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.217+0000 7fb04887b640 1 -- start start 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb04887b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0440b12d0 0x7fb0440cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb04887b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0440b16b0 0x7fb04414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb04887b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb04414f410 0x7fb0440c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb04887b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb04409b7c0 con 0x7fb0440b12d0 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb04887b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb04409b4e0 con 0x7fb04414f410 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb04887b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb04409bce0 con 0x7fb0440b16b0 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb041d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0440b16b0 0x7fb04414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb041d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0440b16b0 0x7fb04414eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:50004/0 (socket says 172.21.15.135:50004) 2024-04-17T06:09:41.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb041d9b640 1 -- 172.21.15.135:0/4144521952 learned_addr learned my addr 172.21.15.135:0/4144521952 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.219+0000 7fb04259c640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0440b12d0 0x7fb0440cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.220+0000 7fb041d9b640 1 -- 172.21.15.135:0/4144521952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb04414f410 msgr2=0x7fb0440c9760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.220+0000 7fb041d9b640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb04414f410 0x7fb0440c9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.220+0000 7fb041d9b640 1 -- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0440b12d0 msgr2=0x7fb0440cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.220+0000 7fb041d9b640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0440b12d0 0x7fb0440cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.220+0000 7fb041d9b640 1 -- 172.21.15.135:0/4144521952 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0440cc560 con 0x7fb0440b16b0 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.220+0000 7fb04259c640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0440b12d0 0x7fb0440cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.220+0000 7fb041d9b640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0440b16b0 0x7fb04414eed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb03800c730 tx=0x7fb03800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.222+0000 7fb032ffd640 1 -- 172.21.15.135:0/4144521952 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038018070 con 0x7fb0440b16b0 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.222+0000 7fb04887b640 1 -- 172.21.15.135:0/4144521952 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0440b64f0 con 0x7fb0440b16b0 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.222+0000 7fb04887b640 1 -- 172.21.15.135:0/4144521952 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0440b6ab0 con 0x7fb0440b16b0 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.222+0000 7fb032ffd640 1 -- 172.21.15.135:0/4144521952 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb038004ce0 con 0x7fb0440b16b0 2024-04-17T06:09:41.223 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.222+0000 7fb032ffd640 1 -- 172.21.15.135:0/4144521952 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03800a860 con 0x7fb0440b16b0 2024-04-17T06:09:41.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.226+0000 7fb032ffd640 1 -- 172.21.15.135:0/4144521952 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb03801b070 con 0x7fb0440b16b0 2024-04-17T06:09:41.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.227+0000 7fb032ffd640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb02406c9c0 0x7fb02406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.227+0000 7fb032ffd640 1 -- 172.21.15.135:0/4144521952 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb03808b2c0 con 0x7fb0440b16b0 2024-04-17T06:09:41.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.227+0000 7fb04259c640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb02406c9c0 0x7fb02406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.228+0000 7fb04259c640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb02406c9c0 0x7fb02406ee80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb03c035000 tx=0x7fb03c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.228+0000 7fb04887b640 1 -- 172.21.15.135:0/4144521952 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb028005290 con 0x7fb0440b16b0 2024-04-17T06:09:41.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.234+0000 7fb032ffd640 1 -- 172.21.15.135:0/4144521952 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb038058d40 con 0x7fb0440b16b0 2024-04-17T06:09:41.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.253+0000 7f8607d31640 1 Processor -- start 2024-04-17T06:09:41.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.253+0000 7f8607d31640 1 -- start start 2024-04-17T06:09:41.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8607d31640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f860015ab80 0x7f860006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8607d31640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f860006c0a0 0x7f86000cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8607d31640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86000affb0 con 0x7f86000af750 2024-04-17T06:09:41.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8607d31640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8600072d20 con 0x7f860015ab80 2024-04-17T06:09:41.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8607d31640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8600071f60 con 0x7f860006c0a0 2024-04-17T06:09:41.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8605acd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f860015ab80 0x7f860006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8605acd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f860015ab80 0x7f860006df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48904/0 (socket says 172.21.15.135:48904) 2024-04-17T06:09:41.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8605acd640 1 -- 172.21.15.135:0/2972096119 learned_addr learned my addr 172.21.15.135:0/2972096119 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f86062ce640 1 --2- 172.21.15.135:0/2972096119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f860006c0a0 0x7f86000cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8605acd640 1 -- 172.21.15.135:0/2972096119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f860006c0a0 msgr2=0x7f86000cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8605acd640 1 --2- 172.21.15.135:0/2972096119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f860006c0a0 0x7f86000cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8605acd640 1 -- 172.21.15.135:0/2972096119 >> v1:172.21.15.190:6789/0 conn(0x7f86000af750 legacy=0x7f860006b990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T06:09:41.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.255+0000 7f8605acd640 1 -- 172.21.15.135:0/2972096119 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600072240 con 0x7f860015ab80 2024-04-17T06:09:41.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.257+0000 7f8605acd640 1 --2- 172.21.15.135:0/2972096119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860015ab80 0x7f860006df20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f85fc008870 tx=0x7f85fc02df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c12d1673e98dcde server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.257+0000 7f8604acb640 1 -- 172.21.15.135:0/2972096119 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc034070 con 0x7f860015ab80 2024-04-17T06:09:41.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.257+0000 7f8604acb640 1 -- 172.21.15.135:0/2972096119 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85fc02ecd0 con 0x7f860015ab80 2024-04-17T06:09:41.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.257+0000 7f8604acb640 1 -- 172.21.15.135:0/2972096119 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc008c00 con 0x7f860015ab80 2024-04-17T06:09:41.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.257+0000 7f111010d640 1 Processor -- start 2024-04-17T06:09:41.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.258+0000 7f111010d640 1 -- start start 2024-04-17T06:09:41.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.258+0000 7f177c4c5640 1 Processor -- start 2024-04-17T06:09:41.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.258+0000 7f177c4c5640 1 -- start start 2024-04-17T06:09:41.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f111010d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11000501a0 0x7f1100054270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f111010d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f110000aaa0 0x7f11000382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f111010d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1100006af0 con 0x7f11000df420 2024-04-17T06:09:41.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f111010d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f11000068a0 con 0x7f11000501a0 2024-04-17T06:09:41.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f111010d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1100006dd0 con 0x7f110000aaa0 2024-04-17T06:09:41.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f8607d31640 1 -- 172.21.15.135:0/2972096119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860015ab80 msgr2=0x7f860006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f8607d31640 1 --2- 172.21.15.135:0/2972096119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860015ab80 0x7f860006df20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f85fc008870 tx=0x7f85fc02df20 comp rx=0 tx=0).stop 2024-04-17T06:09:41.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f8607d31640 1 -- 172.21.15.135:0/2972096119 shutdown_connections 2024-04-17T06:09:41.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f8607d31640 1 --2- 172.21.15.135:0/2972096119 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f860006c0a0 0x7f86000cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f8607d31640 1 --2- 172.21.15.135:0/2972096119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860015ab80 0x7f860006df20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f8607d31640 1 -- 172.21.15.135:0/2972096119 >> 172.21.15.135:0/2972096119 conn(0x7f86000b2290 msgr2=0x7f86000b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f8607d31640 1 -- 172.21.15.135:0/2972096119 shutdown_connections 2024-04-17T06:09:41.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f110dea9640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f11000df420 0x7f110004fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39404/0 (socket says 172.21.15.135:39404) 2024-04-17T06:09:41.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f110dea9640 1 -- 172.21.15.135:0/2929930479 learned_addr learned my addr 172.21.15.135:0/2929930479 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:41.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f110d6a8640 1 --2- 172.21.15.135:0/2929930479 >> v2:172.21.15.190:3300/0 conn(0x7f11000501a0 0x7f1100054270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.259+0000 7f110e6aa640 1 --2- 172.21.15.135:0/2929930479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f110000aaa0 0x7f11000382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f177c4c5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f176c048100 0x7f176c006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f177c4c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c007460 0x7f176c053e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f177c4c5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176c005ec0 con 0x7f176c054390 2024-04-17T06:09:41.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f177c4c5640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f176c0e7b90 con 0x7f176c048100 2024-04-17T06:09:41.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f177c4c5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f176c0547e0 con 0x7f176c007460 2024-04-17T06:09:41.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f177aa62640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f176c054390 0x7f176c04f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39408/0 (socket says 172.21.15.135:39408) 2024-04-17T06:09:41.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f177aa62640 1 -- 172.21.15.135:0/1594719568 learned_addr learned my addr 172.21.15.135:0/1594719568 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:41.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f8607d31640 1 -- 172.21.15.135:0/2972096119 wait complete. 2024-04-17T06:09:41.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f1779a60640 1 --2- 172.21.15.135:0/1594719568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c007460 0x7f176c053e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f177a261640 1 --2- 172.21.15.135:0/1594719568 >> v2:172.21.15.135:3300/0 conn(0x7f176c048100 0x7f176c006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f1779a60640 1 -- 172.21.15.135:0/1594719568 >> v2:172.21.15.135:3300/0 conn(0x7f176c048100 msgr2=0x7f176c006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f1779a60640 1 --2- 172.21.15.135:0/1594719568 >> v2:172.21.15.135:3300/0 conn(0x7f176c048100 0x7f176c006f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f1779a60640 1 -- 172.21.15.135:0/1594719568 >> v1:172.21.15.190:6789/0 conn(0x7f176c054390 legacy=0x7f176c04f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.261+0000 7f1779a60640 1 -- 172.21.15.135:0/1594719568 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c053280 con 0x7f176c007460 2024-04-17T06:09:41.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f110d6a8640 1 -- 172.21.15.135:0/2929930479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f110000aaa0 msgr2=0x7f11000382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f110d6a8640 1 --2- 172.21.15.135:0/2929930479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f110000aaa0 0x7f11000382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f110d6a8640 1 -- 172.21.15.135:0/2929930479 >> v1:172.21.15.190:6789/0 conn(0x7f11000df420 legacy=0x7f110004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.260+0000 7f110d6a8640 1 -- 172.21.15.135:0/2929930479 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11000070b0 con 0x7f11000501a0 2024-04-17T06:09:41.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.261+0000 7f73ce17c640 1 Processor -- start 2024-04-17T06:09:41.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.261+0000 7f6881e14640 1 Processor -- start 2024-04-17T06:09:41.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.261+0000 7f908b4f7640 1 Processor -- start 2024-04-17T06:09:41.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.261+0000 7f73ce17c640 1 -- start start 2024-04-17T06:09:41.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.261+0000 7f8607d31640 1 Processor -- start 2024-04-17T06:09:41.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.261+0000 7f6881e14640 1 -- start start 2024-04-17T06:09:41.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f8607d31640 1 -- start start 2024-04-17T06:09:41.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f1779a60640 1 --2- 172.21.15.135:0/1594719568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c007460 0x7f176c053e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17640097a0 tx=0x7f176402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4748649026641157 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.261+0000 7f908b4f7640 1 -- start start 2024-04-17T06:09:41.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f110d6a8640 1 --2- 172.21.15.135:0/2929930479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11000501a0 0x7f1100054270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1108000b60 tx=0x7f11080a7460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74fc6347f7c44b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f177925f640 1 -- 172.21.15.135:0/1594719568 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764034070 con 0x7f176c007460 2024-04-17T06:09:41.270 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f177925f640 1 -- 172.21.15.135:0/1594719568 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1764030800 con 0x7f176c007460 2024-04-17T06:09:41.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f177925f640 1 -- 172.21.15.135:0/1594719568 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764030bd0 con 0x7f176c007460 2024-04-17T06:09:41.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f6881e14640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f687c162d80 0x7f687c09b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f6881e14640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f687c0d0280 0x7f687c0c5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f6881e14640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687c0d09f0 con 0x7f687c0c5bc0 2024-04-17T06:09:41.271 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f6881e14640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f687c0cce10 con 0x7f687c162d80 2024-04-17T06:09:41.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73ce17c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73c815ab80 0x7f73c806df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f6881e14640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f687c0cd0f0 con 0x7f687c0d0280 2024-04-17T06:09:41.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f8607d31640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860006c0a0 0x7f860014cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f8605acd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860006c0a0 0x7f860014cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f8605acd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860006c0a0 0x7f860014cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48914/0 (socket says 172.21.15.135:48914) 2024-04-17T06:09:41.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f8605acd640 1 -- 172.21.15.135:0/1399195512 learned_addr learned my addr 172.21.15.135:0/1399195512 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f687bfff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f687c0c5bc0 0x7f687c1669d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37810/0 (socket says 172.21.15.135:37810) 2024-04-17T06:09:41.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f687b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f687c162d80 0x7f687c09b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f687b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f687c162d80 0x7f687c09b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36662/0 (socket says 172.21.15.135:36662) 2024-04-17T06:09:41.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f687bfff640 1 -- 172.21.15.135:0/324663148 learned_addr learned my addr 172.21.15.135:0/324663148 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:41.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f110cea7640 1 -- 172.21.15.135:0/2929930479 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080b3070 con 0x7f11000501a0 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f110cea7640 1 -- 172.21.15.135:0/2929930479 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11080ac7d0 con 0x7f11000501a0 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.262+0000 7f110cea7640 1 -- 172.21.15.135:0/2929930479 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080acc60 con 0x7f11000501a0 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f111010d640 1 -- 172.21.15.135:0/2929930479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11000501a0 msgr2=0x7f1100054270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f111010d640 1 --2- 172.21.15.135:0/2929930479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11000501a0 0x7f1100054270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1108000b60 tx=0x7f11080a7460 comp rx=0 tx=0).stop 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73c77fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73c815ab80 0x7f73c806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73c77fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73c815ab80 0x7f73c806df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36650/0 (socket says 172.21.15.135:36650) 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73c77fe640 1 -- 172.21.15.135:0/2654404634 learned_addr learned my addr 172.21.15.135:0/2654404634 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2654404634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 0x7f73c80cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73ce17c640 1 -- 172.21.15.135:0/2654404634 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73c80affb0 con 0x7f73c80af750 2024-04-17T06:09:41.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73ce17c640 1 -- 172.21.15.135:0/2654404634 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f73c8072d20 con 0x7f73c815ab80 2024-04-17T06:09:41.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73ce17c640 1 -- 172.21.15.135:0/2654404634 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f73c8071f60 con 0x7f73c806c0a0 2024-04-17T06:09:41.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f73c7fff640 1 --2- 172.21.15.135:0/2654404634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 0x7f73c80cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f177c4c5640 1 -- 172.21.15.135:0/1594719568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c007460 msgr2=0x7f176c053e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1594719568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c007460 0x7f176c053e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17640097a0 tx=0x7f176402b940 comp rx=0 tx=0).stop 2024-04-17T06:09:41.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f8607d31640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86000af750 0x7f860014e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f8607d31640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f860014e8c0 0x7f8600155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f908b4f7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f907c036130 0x7f907c038750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f908b4f7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c0545f0 con 0x7f907c0e1210 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f908b4f7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c039320 con 0x7f907c038c90 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f908b4f7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f907c036590 con 0x7f907c036130 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f9089cf4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f907c036130 0x7f907c038750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f9089cf4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f907c036130 0x7f907c038750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48922/0 (socket says 172.21.15.135:48922) 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f9089cf4640 1 -- 172.21.15.135:0/1396663606 learned_addr learned my addr 172.21.15.135:0/1396663606 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f9089cf4640 1 -- 172.21.15.135:0/1396663606 >> v1:172.21.15.190:6789/0 conn(0x7f907c038c90 legacy=0x7f907c039f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T06:09:41.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f9089cf4640 1 -- 172.21.15.135:0/1396663606 >> v1:172.21.15.135:6789/0 conn(0x7f907c0e1210 legacy=0x7f907c051da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T06:09:41.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86000afc10 con 0x7f860006c0a0 2024-04-17T06:09:41.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8600072800 con 0x7f860014e8c0 2024-04-17T06:09:41.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f86000afd90 con 0x7f86000af750 2024-04-17T06:09:41.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.263+0000 7f111010d640 1 -- 172.21.15.135:0/2929930479 shutdown_connections 2024-04-17T06:09:41.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f111010d640 1 --2- 172.21.15.135:0/2929930479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f110000aaa0 0x7f11000382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f111010d640 1 --2- 172.21.15.135:0/2929930479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11000501a0 0x7f1100054270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f111010d640 1 -- 172.21.15.135:0/2929930479 >> 172.21.15.135:0/2929930479 conn(0x7f1100009fb0 msgr2=0x7f110000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f177c4c5640 1 -- 172.21.15.135:0/1594719568 shutdown_connections 2024-04-17T06:09:41.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1594719568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c007460 0x7f176c053e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1594719568 >> v2:172.21.15.135:3300/0 conn(0x7f176c048100 0x7f176c006f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f177c4c5640 1 -- 172.21.15.135:0/1594719568 >> 172.21.15.135:0/1594719568 conn(0x7f176c009fe0 msgr2=0x7f176c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f86062ce640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f860014e8c0 0x7f8600155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f177c4c5640 1 -- 172.21.15.135:0/1594719568 shutdown_connections 2024-04-17T06:09:41.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807171501 0 0) 0x7f687c0d09f0 con 0x7f687c0c5bc0 2024-04-17T06:09:41.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6864003520 con 0x7f687c0c5bc0 2024-04-17T06:09:41.279 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f111010d640 1 -- 172.21.15.135:0/2929930479 shutdown_connections 2024-04-17T06:09:41.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f177c4c5640 1 -- 172.21.15.135:0/1594719568 wait complete. 2024-04-17T06:09:41.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f86062ce640 1 -- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86000af750 msgr2=0x7f860014e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f86062ce640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86000af750 0x7f860014e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f86062ce640 1 -- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860006c0a0 msgr2=0x7f860014cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f86062ce640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860006c0a0 0x7f860014cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.280 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f86062ce640 1 -- 172.21.15.135:0/1399195512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860015af60 con 0x7f860014e8c0 2024-04-17T06:09:41.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4114818101 0 0) 0x7f6864003520 con 0x7f687c0c5bc0 2024-04-17T06:09:41.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f687c0d0280 msgr2=0x7f687c0c5680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-17T06:09:41.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f687c0d0280 0x7f687c0c5680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 >> v2:172.21.15.190:3300/0 conn(0x7f687c162d80 msgr2=0x7f687c09b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 --2- 172.21.15.135:0/324663148 >> v2:172.21.15.190:3300/0 conn(0x7f687c162d80 0x7f687c09b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c0cd3d0 con 0x7f687c0c5bc0 2024-04-17T06:09:41.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f8605acd640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860006c0a0 0x7f860014cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:41.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f68740019f0 con 0x7f687c0c5bc0 2024-04-17T06:09:41.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 >> v1:172.21.15.135:6789/0 conn(0x7f687c0c5bc0 legacy=0x7f687c1669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68640039b0 0x7f6864003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864004800 0x7f6864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68640051c0 0x7f686400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f687c0cce10 con 0x7f6864004800 2024-04-17T06:09:41.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f687c0cd0f0 con 0x7f68640039b0 2024-04-17T06:09:41.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f686400c0b0 con 0x7f68640051c0 2024-04-17T06:09:41.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f73c7fff640 1 -- 172.21.15.135:0/2654404634 >> v2:172.21.15.190:3300/0 conn(0x7f73c815ab80 msgr2=0x7f73c806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f86062ce640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f860014e8c0 0x7f8600155ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d530 tx=0x7f85f000d9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f73c7fff640 1 --2- 172.21.15.135:0/2654404634 >> v2:172.21.15.190:3300/0 conn(0x7f73c815ab80 0x7f73c806df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f73c7fff640 1 -- 172.21.15.135:0/2654404634 >> v1:172.21.15.190:6789/0 conn(0x7f73c80af750 legacy=0x7f73c806b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f73c7fff640 1 -- 172.21.15.135:0/2654404634 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c8072240 con 0x7f73c806c0a0 2024-04-17T06:09:41.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f111010d640 1 -- 172.21.15.135:0/2929930479 wait complete. 2024-04-17T06:09:41.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.264+0000 7f9089cf4640 1 -- 172.21.15.135:0/1396663606 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c009c00 con 0x7f907c036130 2024-04-17T06:09:41.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f9089cf4640 1 --2- 172.21.15.135:0/1396663606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c036130 0x7f907c038750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9074013150 tx=0x7f9074034120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806393f01181318 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f90894f3640 1 -- 172.21.15.135:0/1396663606 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740110d0 con 0x7f907c036130 2024-04-17T06:09:41.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f90894f3640 1 -- 172.21.15.135:0/1396663606 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9074011270 con 0x7f907c036130 2024-04-17T06:09:41.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f85f67fc640 1 -- 172.21.15.135:0/1399195512 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f00125f0 con 0x7f860014e8c0 2024-04-17T06:09:41.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860015b240 con 0x7f860014e8c0 2024-04-17T06:09:41.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f860015b800 con 0x7f860014e8c0 2024-04-17T06:09:41.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f85f67fc640 1 -- 172.21.15.135:0/1399195512 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f0019070 con 0x7f860014e8c0 2024-04-17T06:09:41.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f85f67fc640 1 -- 172.21.15.135:0/1399195512 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f00129d0 con 0x7f860014e8c0 2024-04-17T06:09:41.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f687bfff640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68640051c0 0x7f686400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f687affd640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864004800 0x7f6864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f687bfff640 1 -- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68640039b0 msgr2=0x7f6864003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-17T06:09:41.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f687bfff640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68640039b0 0x7f6864003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f687bfff640 1 -- 172.21.15.135:0/324663148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864004800 msgr2=0x7f6864004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f687bfff640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864004800 0x7f6864004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f687bfff640 1 -- 172.21.15.135:0/324663148 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c0cd3d0 con 0x7f68640051c0 2024-04-17T06:09:41.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.265+0000 7f687affd640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6864004800 0x7f6864004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:41.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f111010d640 1 Processor -- start 2024-04-17T06:09:41.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f177c4c5640 1 Processor -- start 2024-04-17T06:09:41.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f177c4c5640 1 -- start start 2024-04-17T06:09:41.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f111010d640 1 -- start start 2024-04-17T06:09:41.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f90894f3640 1 -- 172.21.15.135:0/1396663606 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074011640 con 0x7f907c036130 2024-04-17T06:09:41.287 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f687bfff640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68640051c0 0x7f686400b9c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f687400a900 tx=0x7f6864005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a0d37414b18d9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687400a500 con 0x7f68640051c0 2024-04-17T06:09:41.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6874034650 con 0x7f68640051c0 2024-04-17T06:09:41.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f687a7fc640 1 -- 172.21.15.135:0/324663148 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874034a40 con 0x7f68640051c0 2024-04-17T06:09:41.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.266+0000 7f73c7fff640 1 --2- 172.21.15.135:0/2654404634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 0x7f73c80cd740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009d90 tx=0x7f73c002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37add4ca0c239ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f73c67fc640 1 -- 172.21.15.135:0/2654404634 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0039070 con 0x7f73c806c0a0 2024-04-17T06:09:41.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f73c67fc640 1 -- 172.21.15.135:0/2654404634 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c002cbd0 con 0x7f73c806c0a0 2024-04-17T06:09:41.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f73c67fc640 1 -- 172.21.15.135:0/2654404634 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0034440 con 0x7f73c806c0a0 2024-04-17T06:09:41.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f177c4c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c007460 0x7f176c173120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f177c4c5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f176c048100 0x7f176c0e3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f908b4f7640 1 -- 172.21.15.135:0/1396663606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c036130 msgr2=0x7f907c038750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f908b4f7640 1 --2- 172.21.15.135:0/1396663606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c036130 0x7f907c038750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9074013150 tx=0x7f9074034120 comp rx=0 tx=0).stop 2024-04-17T06:09:41.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f177c4c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c054390 0x7f176c0367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f177c4c5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f176c006440 con 0x7f176c048100 2024-04-17T06:09:41.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f177aa62640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c054390 0x7f176c0367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f111010d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f110000aaa0 0x7f11000cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f177aa62640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c054390 0x7f176c0367c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:50072/0 (socket says 172.21.15.135:50072) 2024-04-17T06:09:41.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f177aa62640 1 -- 172.21.15.135:0/1455361815 learned_addr learned my addr 172.21.15.135:0/1455361815 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f111010d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11000501a0 0x7f11000cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.267+0000 7f177c4c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f176c006a90 con 0x7f176c007460 2024-04-17T06:09:41.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f110dea9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f110000aaa0 0x7f11000cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f176c006720 con 0x7f176c054390 2024-04-17T06:09:41.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f110dea9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f110000aaa0 0x7f11000cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36698/0 (socket says 172.21.15.135:36698) 2024-04-17T06:09:41.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f177a261640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c007460 0x7f176c173120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f73ce17c640 1 -- 172.21.15.135:0/2654404634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 msgr2=0x7f73c80cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2654404634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 0x7f73c80cd740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009d90 tx=0x7f73c002be20 comp rx=0 tx=0).stop 2024-04-17T06:09:41.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f6881e14640 1 -- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68640051c0 msgr2=0x7f686400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f6881e14640 1 --2- 172.21.15.135:0/324663148 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68640051c0 0x7f686400b9c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f687400a900 tx=0x7f6864005830 comp rx=0 tx=0).stop 2024-04-17T06:09:41.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f687bfff640 1 -- 172.21.15.135:0/324663148 reap_dead start 2024-04-17T06:09:41.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f73ce17c640 1 -- 172.21.15.135:0/2654404634 shutdown_connections 2024-04-17T06:09:41.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2654404634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 0x7f73c80cd740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2654404634 >> v2:172.21.15.190:3300/0 conn(0x7f73c815ab80 0x7f73c806df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f73ce17c640 1 -- 172.21.15.135:0/2654404634 >> 172.21.15.135:0/2654404634 conn(0x7f73c80b2290 msgr2=0x7f73c80b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f6881e14640 1 -- 172.21.15.135:0/324663148 shutdown_connections 2024-04-17T06:09:41.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f6881e14640 1 -- 172.21.15.135:0/324663148 >> 172.21.15.135:0/324663148 conn(0x7f687c0b41a0 msgr2=0x7f687c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f110dea9640 1 -- 172.21.15.135:0/1672613412 learned_addr learned my addr 172.21.15.135:0/1672613412 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f73ce17c640 1 -- 172.21.15.135:0/2654404634 shutdown_connections 2024-04-17T06:09:41.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f6881e14640 1 -- 172.21.15.135:0/324663148 shutdown_connections 2024-04-17T06:09:41.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f111010d640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000df420 0x7f11000d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1100052b40 con 0x7f11000501a0 2024-04-17T06:09:41.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f11000547b0 con 0x7f110000aaa0 2024-04-17T06:09:41.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1100052cc0 con 0x7f11000df420 2024-04-17T06:09:41.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f73ce17c640 1 -- 172.21.15.135:0/2654404634 wait complete. 2024-04-17T06:09:41.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f6881e14640 1 -- 172.21.15.135:0/324663148 wait complete. 2024-04-17T06:09:41.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f110d6a8640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11000501a0 0x7f11000cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f1779a60640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f176c048100 0x7f176c0e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.268+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e8005290 con 0x7f860014e8c0 2024-04-17T06:09:41.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f110dea9640 1 -- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000df420 msgr2=0x7f11000d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f177aa62640 1 -- 172.21.15.135:0/1455361815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c007460 msgr2=0x7f176c173120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f177aa62640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c007460 0x7f176c173120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f177aa62640 1 -- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f176c048100 msgr2=0x7f176c0e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f177aa62640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f176c048100 0x7f176c0e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f177aa62640 1 -- 172.21.15.135:0/1455361815 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c172360 con 0x7f176c054390 2024-04-17T06:09:41.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f85f67fc640 1 -- 172.21.15.135:0/1399195512 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85f0012b70 con 0x7f860014e8c0 2024-04-17T06:09:41.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f110e6aa640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000df420 0x7f11000d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f110dea9640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000df420 0x7f11000d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f110dea9640 1 -- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11000501a0 msgr2=0x7f11000cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f110dea9640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11000501a0 0x7f11000cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f110dea9640 1 -- 172.21.15.135:0/1672613412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11000d0200 con 0x7f110000aaa0 2024-04-17T06:09:41.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f110d6a8640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11000501a0 0x7f11000cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:41.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f110e6aa640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000df420 0x7f11000d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:41.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f6881e14640 1 Processor -- start 2024-04-17T06:09:41.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f177aa62640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c054390 0x7f176c0367c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1774063580 tx=0x7f1774069500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f110dea9640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f110000aaa0 0x7f11000cfb70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c480 tx=0x7f10f800c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f85f67fc640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f85e406c940 0x7f85e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f176affd640 1 -- 172.21.15.135:0/1455361815 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177406b990 con 0x7f176c054390 2024-04-17T06:09:41.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f176affd640 1 -- 172.21.15.135:0/1455361815 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1774090020 con 0x7f176c054390 2024-04-17T06:09:41.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f8605acd640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f85e406c940 0x7f85e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f85f67fc640 1 -- 172.21.15.135:0/1399195512 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f85f008c110 con 0x7f860014e8c0 2024-04-17T06:09:41.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f73ce17c640 1 Processor -- start 2024-04-17T06:09:41.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f73ce17c640 1 -- start start 2024-04-17T06:09:41.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f176affd640 1 -- 172.21.15.135:0/1455361815 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177406bdf0 con 0x7f176c054390 2024-04-17T06:09:41.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.270+0000 7f6881e14640 1 -- start start 2024-04-17T06:09:41.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f8605acd640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f85e406c940 0x7f85e406ee00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f85fc02e4b0 tx=0x7f85fc034000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73ce17c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 0x7f73c80d0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73ce17c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c80af750 0x7f73c815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73ce17c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73c815ab80 0x7f73c815c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73ce17c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73c80afc10 con 0x7f73c815ab80 2024-04-17T06:09:41.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73ce17c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f73c8072800 con 0x7f73c80af750 2024-04-17T06:09:41.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73ce17c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f73c80afd90 con 0x7f73c806c0a0 2024-04-17T06:09:41.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c172640 con 0x7f176c054390 2024-04-17T06:09:41.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c0e3640 con 0x7f176c054390 2024-04-17T06:09:41.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73c6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c80af750 0x7f73c815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f6881e14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f687c06dd60 0x7f687c1fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73c6ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c80af750 0x7f73c815bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36712/0 (socket says 172.21.15.135:36712) 2024-04-17T06:09:41.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f73c6ffd640 1 -- 172.21.15.135:0/2428985989 learned_addr learned my addr 172.21.15.135:0/2428985989 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f73c7fff640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73c815ab80 0x7f73c815c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f6881e14640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c06e140 0x7f687c0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f6881e14640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f687c0c8830 0x7f687c1f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f6881e14640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f687c163160 con 0x7f687c06dd60 2024-04-17T06:09:41.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f6881e14640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f687c0d0cd0 con 0x7f687c06e140 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f6881e14640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f687c0ce8e0 con 0x7f687c0c8830 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c06e140 0x7f687c0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687affd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c06e140 0x7f687c0c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:36714/0 (socket says 172.21.15.135:36714) 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687affd640 1 -- 172.21.15.135:0/824555902 learned_addr learned my addr 172.21.15.135:0/824555902 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687b7fe640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f687c06dd60 0x7f687c1fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f73c6ffd640 1 -- 172.21.15.135:0/2428985989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 msgr2=0x7f73c80d0770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f73c6ffd640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 0x7f73c80d0770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f73c6ffd640 1 -- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73c815ab80 msgr2=0x7f73c815c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f73c6ffd640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73c815ab80 0x7f73c815c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f73c6ffd640 1 -- 172.21.15.135:0/2428985989 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c815ca60 con 0x7f73c80af750 2024-04-17T06:09:41.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687b7fe640 1 -- 172.21.15.135:0/824555902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f687c0c8830 msgr2=0x7f687c1f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687b7fe640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f687c0c8830 0x7f687c1f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687b7fe640 1 -- 172.21.15.135:0/824555902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c06e140 msgr2=0x7f687c0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687b7fe640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c06e140 0x7f687c0c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687b7fe640 1 -- 172.21.15.135:0/824555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c1fd010 con 0x7f687c06dd60 2024-04-17T06:09:41.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f73c6ffd640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c80af750 0x7f73c815bfe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b3f0 tx=0x7f73b800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.272+0000 7f687b7fe640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f687c06dd60 0x7f687c1fc170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f686c00d560 tx=0x7f686c00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f908b4f7640 1 -- 172.21.15.135:0/1396663606 shutdown_connections 2024-04-17T06:09:41.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f908b4f7640 1 --2- 172.21.15.135:0/1396663606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c036130 0x7f907c038750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f908b4f7640 1 -- 172.21.15.135:0/1396663606 >> 172.21.15.135:0/1396663606 conn(0x7f907c053750 msgr2=0x7f907c053b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f908b4f7640 1 -- 172.21.15.135:0/1396663606 shutdown_connections 2024-04-17T06:09:41.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.269+0000 7f908b4f7640 1 -- 172.21.15.135:0/1396663606 wait complete. 2024-04-17T06:09:41.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f908b4f7640 1 Processor -- start 2024-04-17T06:09:41.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.271+0000 7f908b4f7640 1 -- start start 2024-04-17T06:09:41.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908b4f7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907c036130 0x7f907c0dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908b4f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c038c90 0x7f907c0e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f6863fff640 1 -- 172.21.15.135:0/824555902 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0125f0 con 0x7f687c06dd60 2024-04-17T06:09:41.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f176affd640 1 -- 172.21.15.135:0/1455361815 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f177406d110 con 0x7f176c054390 2024-04-17T06:09:41.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908a4f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907c036130 0x7f907c0dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908a4f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907c036130 0x7f907c0dbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:50122/0 (socket says 172.21.15.135:50122) 2024-04-17T06:09:41.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f9089cf4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c038c90 0x7f907c0e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f9089cf4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c038c90 0x7f907c0e3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48980/0 (socket says 172.21.15.135:48980) 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908a4f5640 1 -- 172.21.15.135:0/1595346224 learned_addr learned my addr 172.21.15.135:0/1595346224 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908b4f7640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907c0e1210 0x7f907c0d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908b4f7640 1 -- 172.21.15.135:0/1595346224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0518a0 con 0x7f907c038c90 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908b4f7640 1 -- 172.21.15.135:0/1595346224 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0515c0 con 0x7f907c0e1210 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f908b4f7640 1 -- 172.21.15.135:0/1595346224 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f907c0524b0 con 0x7f907c036130 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c1fca50 con 0x7f687c06dd60 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c0caa60 con 0x7f687c06dd60 2024-04-17T06:09:41.311 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f6863fff640 1 -- 172.21.15.135:0/824555902 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c019070 con 0x7f687c06dd60 2024-04-17T06:09:41.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.273+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176c0053c0 con 0x7f176c054390 2024-04-17T06:09:41.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f6863fff640 1 -- 172.21.15.135:0/824555902 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0129d0 con 0x7f687c06dd60 2024-04-17T06:09:41.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f176affd640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f175806c970 0x7f175806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f176affd640 1 -- 172.21.15.135:0/1455361815 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1774106b50 con 0x7f176c054390 2024-04-17T06:09:41.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f177a261640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f175806c970 0x7f175806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f9089cf4640 1 -- 172.21.15.135:0/1595346224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907c036130 msgr2=0x7f907c0dbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f9089cf4640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907c036130 0x7f907c0dbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f9089cf4640 1 -- 172.21.15.135:0/1595346224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907c0e1210 msgr2=0x7f907c0d3cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T06:09:41.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f9089cf4640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907c0e1210 0x7f907c0d3cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f9089cf4640 1 -- 172.21.15.135:0/1595346224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c0e3a70 con 0x7f907c038c90 2024-04-17T06:09:41.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.274+0000 7f9089cf4640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c038c90 0x7f907c0e3530 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f907403a000 tx=0x7f90740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f907affd640 1 -- 172.21.15.135:0/1595346224 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907403cda0 con 0x7f907c038c90 2024-04-17T06:09:41.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f907affd640 1 -- 172.21.15.135:0/1595346224 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9074045400 con 0x7f907c038c90 2024-04-17T06:09:41.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f907affd640 1 -- 172.21.15.135:0/1595346224 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740457b0 con 0x7f907c038c90 2024-04-17T06:09:41.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c005290 con 0x7f687c06dd60 2024-04-17T06:09:41.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f908b4f7640 1 -- 172.21.15.135:0/1595346224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c0dc2e0 con 0x7f907c038c90 2024-04-17T06:09:41.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f908b4f7640 1 -- 172.21.15.135:0/1595346224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c0dc8a0 con 0x7f907c038c90 2024-04-17T06:09:41.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f85f67fc640 1 -- 172.21.15.135:0/1399195512 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85f005a270 con 0x7f860014e8c0 2024-04-17T06:09:41.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f6863fff640 1 -- 172.21.15.135:0/824555902 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f686c012b70 con 0x7f687c06dd60 2024-04-17T06:09:41.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f6863fff640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f686806c8f0 0x7f686806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.315 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f6863fff640 1 -- 172.21.15.135:0/824555902 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f686c08c1a0 con 0x7f687c06dd60 2024-04-17T06:09:41.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f10fe7fc640 1 -- 172.21.15.135:0/1672613412 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8017070 con 0x7f110000aaa0 2024-04-17T06:09:41.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f10fe7fc640 1 -- 172.21.15.135:0/1672613412 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f8005ce0 con 0x7f110000aaa0 2024-04-17T06:09:41.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.275+0000 7f10fe7fc640 1 -- 172.21.15.135:0/1672613412 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f80072f0 con 0x7f110000aaa0 2024-04-17T06:09:41.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f73affff640 1 -- 172.21.15.135:0/2428985989 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8015020 con 0x7f73c80af750 2024-04-17T06:09:41.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11000d04e0 con 0x7f110000aaa0 2024-04-17T06:09:41.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11000d0aa0 con 0x7f110000aaa0 2024-04-17T06:09:41.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c815cd10 con 0x7f73c80af750 2024-04-17T06:09:41.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c815d2d0 con 0x7f73c80af750 2024-04-17T06:09:41.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.278+0000 7f73affff640 1 -- 172.21.15.135:0/2428985989 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b8011830 con 0x7f73c80af750 2024-04-17T06:09:41.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.279+0000 7f73affff640 1 -- 172.21.15.135:0/2428985989 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8011c00 con 0x7f73c80af750 2024-04-17T06:09:41.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.276+0000 7f9078ff9640 1 -- 172.21.15.135:0/1595346224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9060005290 con 0x7f907c038c90 2024-04-17T06:09:41.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.279+0000 7f907affd640 1 -- 172.21.15.135:0/1595346224 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9074045950 con 0x7f907c038c90 2024-04-17T06:09:41.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.279+0000 7f176affd640 1 -- 172.21.15.135:0/1455361815 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f177409a030 con 0x7f176c054390 2024-04-17T06:09:41.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.279+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a8005290 con 0x7f73c80af750 2024-04-17T06:09:41.318 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.280+0000 7f73affff640 1 -- 172.21.15.135:0/2428985989 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73b8018020 con 0x7f73c80af750 2024-04-17T06:09:41.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.279+0000 7f907affd640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f906806c9c0 0x7f906806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.279+0000 7f10fe7fc640 1 -- 172.21.15.135:0/1672613412 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10f801a070 con 0x7f110000aaa0 2024-04-17T06:09:41.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.279+0000 7f907affd640 1 -- 172.21.15.135:0/1595346224 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f90740ae350 con 0x7f907c038c90 2024-04-17T06:09:41.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.280+0000 7f687affd640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f686806c8f0 0x7f686806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.280+0000 7f10fe7fc640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f10ec06c9c0 0x7f10ec06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.280+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11000089d0 con 0x7f110000aaa0 2024-04-17T06:09:41.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.280+0000 7f10fe7fc640 1 -- 172.21.15.135:0/1672613412 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f10f808b190 con 0x7f110000aaa0 2024-04-17T06:09:41.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.280+0000 7f110d6a8640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f10ec06c9c0 0x7f10ec06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.280+0000 7f73affff640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f73b406c9c0 0x7f73b406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.281+0000 7f73affff640 1 -- 172.21.15.135:0/2428985989 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f73b808bd80 con 0x7f73c80af750 2024-04-17T06:09:41.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.280+0000 7f177a261640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f175806c970 0x7f175806ee30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1770006fd0 tx=0x7f1770008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.281+0000 7f687affd640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f686806c8f0 0x7f686806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f687c06eb90 tx=0x7f6870008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.281+0000 7f73c77fe640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f73b406c9c0 0x7f73b406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.281+0000 7f908a4f5640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f906806c9c0 0x7f906806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.282+0000 7f908a4f5640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f906806c9c0 0x7f906806ee80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9080006fd0 tx=0x7f9080008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.282+0000 7f110d6a8640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f10ec06c9c0 0x7f10ec06ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1100008680 tx=0x7f110806f030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.282+0000 7f73c77fe640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f73b406c9c0 0x7f73b406ee80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0099e0 tx=0x7f73bc009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.284+0000 7f6863fff640 1 -- 172.21.15.135:0/824555902 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f686c05a1f0 con 0x7f687c06dd60 2024-04-17T06:09:41.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.285+0000 7f73affff640 1 -- 172.21.15.135:0/2428985989 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73b801f020 con 0x7f73c80af750 2024-04-17T06:09:41.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.290+0000 7f10fe7fc640 1 -- 172.21.15.135:0/1672613412 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10f8058ca0 con 0x7f110000aaa0 2024-04-17T06:09:41.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.293+0000 7f907affd640 1 -- 172.21.15.135:0/1595346224 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f907407c4b0 con 0x7f907c038c90 2024-04-17T06:09:41.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.313+0000 7f3953eb3640 1 Processor -- start 2024-04-17T06:09:41.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.313+0000 7f3953eb3640 1 -- start start 2024-04-17T06:09:41.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.315+0000 7f3953eb3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f394c06d120 0x7f394c09bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.315+0000 7f3953eb3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f394c09ac40 con 0x7f394c160ba0 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.315+0000 7f3953eb3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f394c09af20 con 0x7f394c0c5d70 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.315+0000 7f3953eb3640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f394c09a9f0 con 0x7f394c06d120 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.315+0000 7f395144e640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f394c0c5d70 0x7f394c0d0260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:39440/0 (socket says 172.21.15.135:39440) 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.315+0000 7f395144e640 1 -- 172.21.15.135:0/4071449940 learned_addr learned my addr 172.21.15.135:0/4071449940 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.315+0000 7f3952450640 1 --2- 172.21.15.135:0/4071449940 >> v2:172.21.15.135:3300/0 conn(0x7f394c06d120 0x7f394c09bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.316+0000 7f3952450640 1 -- 172.21.15.135:0/4071449940 >> v1:172.21.15.190:6789/0 conn(0x7f394c0c5d70 legacy=0x7f394c0d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.316+0000 7f3952450640 1 -- 172.21.15.135:0/4071449940 >> v1:172.21.15.135:6789/0 conn(0x7f394c160ba0 legacy=0x7f394c0c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.316+0000 7f3952450640 1 -- 172.21.15.135:0/4071449940 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c09b200 con 0x7f394c06d120 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.317+0000 7f3952450640 1 --2- 172.21.15.135:0/4071449940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c06d120 0x7f394c09bce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3944009d90 tx=0x7f394402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed26bfbb6159c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.319+0000 7f3950c4d640 1 -- 172.21.15.135:0/4071449940 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944039070 con 0x7f394c06d120 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.319+0000 7f3950c4d640 1 -- 172.21.15.135:0/4071449940 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394402cae0 con 0x7f394c06d120 2024-04-17T06:09:41.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.319+0000 7f3950c4d640 1 -- 172.21.15.135:0/4071449940 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944034440 con 0x7f394c06d120 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.320+0000 7f3953eb3640 1 -- 172.21.15.135:0/4071449940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c06d120 msgr2=0x7f394c09bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.320+0000 7f3953eb3640 1 --2- 172.21.15.135:0/4071449940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c06d120 0x7f394c09bce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3944009d90 tx=0x7f394402bdb0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.320+0000 7f3953eb3640 1 -- 172.21.15.135:0/4071449940 shutdown_connections 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.320+0000 7f3953eb3640 1 --2- 172.21.15.135:0/4071449940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c06d120 0x7f394c09bce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.320+0000 7f3953eb3640 1 -- 172.21.15.135:0/4071449940 >> 172.21.15.135:0/4071449940 conn(0x7f394c0b4180 msgr2=0x7f394c0b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.320+0000 7f3953eb3640 1 -- 172.21.15.135:0/4071449940 shutdown_connections 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.320+0000 7f3953eb3640 1 -- 172.21.15.135:0/4071449940 wait complete. 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.322+0000 7f3953eb3640 1 Processor -- start 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.322+0000 7f3953eb3640 1 -- start start 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.323+0000 7f3953eb3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f394c06d120 0x7f394c08b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.323+0000 7f3953eb3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c0c5d70 0x7f394c0b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f395144e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c0c5d70 0x7f394c0b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f395144e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c0c5d70 0x7f394c0b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49008/0 (socket says 172.21.15.135:49008) 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f395144e640 1 -- 172.21.15.135:0/1438397712 learned_addr learned my addr 172.21.15.135:0/1438397712 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f3953eb3640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f394c160ba0 0x7f394c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f394c163ea0 con 0x7f394c0c5d70 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f394c09b7c0 con 0x7f394c160ba0 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f394c164020 con 0x7f394c06d120 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f395144e640 1 -- 172.21.15.135:0/1438397712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f394c06d120 msgr2=0x7f394c08b280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f395144e640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f394c06d120 0x7f394c08b280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f395144e640 1 -- 172.21.15.135:0/1438397712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f394c160ba0 msgr2=0x7f394c0c9760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f395144e640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f394c160ba0 0x7f394c0c9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.324+0000 7f395144e640 1 -- 172.21.15.135:0/1438397712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c0cbdd0 con 0x7f394c0c5d70 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.325+0000 7f395144e640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c0c5d70 0x7f394c0b0770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f394800c8f0 tx=0x7f394800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.326+0000 7f39427fc640 1 -- 172.21.15.135:0/1438397712 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948018070 con 0x7f394c0c5d70 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.327+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c0cc0b0 con 0x7f394c0c5d70 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.328+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c0aff70 con 0x7f394c0c5d70 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.331+0000 7f39427fc640 1 -- 172.21.15.135:0/1438397712 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39480072d0 con 0x7f394c0c5d70 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.331+0000 7f39427fc640 1 -- 172.21.15.135:0/1438397712 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480047e0 con 0x7f394c0c5d70 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.331+0000 7f39427fc640 1 -- 172.21.15.135:0/1438397712 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f394801b070 con 0x7f394c0c5d70 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.331+0000 7f39427fc640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f393006ca20 0x7f393006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.332+0000 7f3951c4f640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f393006ca20 0x7f393006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.332+0000 7f39427fc640 1 -- 172.21.15.135:0/1438397712 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f394808b460 con 0x7f394c0c5d70 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.333+0000 7f3951c4f640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f393006ca20 0x7f393006eee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f393c001440 tx=0x7f393c007990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.333+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c074940 con 0x7f394c0c5d70 2024-04-17T06:09:41.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.343+0000 7f39427fc640 1 -- 172.21.15.135:0/1438397712 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39480595c0 con 0x7f394c0c5d70 2024-04-17T06:09:41.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.637+0000 7fb04887b640 1 -- 172.21.15.135:0/4144521952 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb028005580 con 0x7fb0440b16b0 2024-04-17T06:09:41.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.638+0000 7fb032ffd640 1 -- 172.21.15.135:0/4144521952 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb03805d040 con 0x7fb0440b16b0 2024-04-17T06:09:41.636 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:41.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.641+0000 7fb030ff9640 1 -- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb02406c9c0 msgr2=0x7fb02406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.641+0000 7fb030ff9640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb02406c9c0 0x7fb02406ee80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb03c035000 tx=0x7fb03c02e000 comp rx=0 tx=0).stop 2024-04-17T06:09:41.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.641+0000 7fb030ff9640 1 -- 172.21.15.135:0/4144521952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0440b16b0 msgr2=0x7fb04414eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0440b16b0 0x7fb04414eed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb03800c730 tx=0x7fb03800cbf0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 -- 172.21.15.135:0/4144521952 shutdown_connections 2024-04-17T06:09:41.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb04414f410 0x7fb0440c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0440b16b0 0x7fb04414eed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb02406c9c0 0x7fb02406ee80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 --2- 172.21.15.135:0/4144521952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb0440b12d0 0x7fb0440cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 -- 172.21.15.135:0/4144521952 >> 172.21.15.135:0/4144521952 conn(0x7fb0440b4180 msgr2=0x7fb044161880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 -- 172.21.15.135:0/4144521952 shutdown_connections 2024-04-17T06:09:41.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.642+0000 7fb030ff9640 1 -- 172.21.15.135:0/4144521952 wait complete. 2024-04-17T06:09:41.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-04-17T06:09:41.658 DEBUG:teuthology.parallel:result is None 2024-04-17T06:09:41.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.675+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f73a8005580 con 0x7f73c80af750 2024-04-17T06:09:41.673 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.676+0000 7f73affff640 1 -- 172.21.15.135:0/2428985989 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f73b8059fa0 con 0x7f73c80af750 2024-04-17T06:09:41.674 INFO:teuthology.orchestra.run.smithi135.stdout:55834574859 2024-04-17T06:09:41.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f73b406c9c0 msgr2=0x7f73b406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f73b406c9c0 0x7f73b406ee80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0099e0 tx=0x7f73bc009240 comp rx=0 tx=0).stop 2024-04-17T06:09:41.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c80af750 msgr2=0x7f73c815bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c80af750 0x7f73c815bfe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b3f0 tx=0x7f73b800b8b0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 shutdown_connections 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f73b406c9c0 0x7f73b406ee80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73c815ab80 0x7f73c815c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f73c80af750 0x7f73c815bfe0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 --2- 172.21.15.135:0/2428985989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73c806c0a0 0x7f73c80d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 >> 172.21.15.135:0/2428985989 conn(0x7f73c80b2290 msgr2=0x7f73c80ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.678+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 shutdown_connections 2024-04-17T06:09:41.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.679+0000 7f73ce17c640 1 -- 172.21.15.135:0/2428985989 wait complete. 2024-04-17T06:09:41.678 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.680+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f85e8005580 con 0x7f860014e8c0 2024-04-17T06:09:41.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.681+0000 7f85f67fc640 1 -- 172.21.15.135:0/1399195512 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f85f0059b70 con 0x7f860014e8c0 2024-04-17T06:09:41.679 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.684+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f85e406c940 msgr2=0x7f85e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.684+0000 7f8607d31640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f85e406c940 0x7f85e406ee00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f85fc02e4b0 tx=0x7f85fc034000 comp rx=0 tx=0).stop 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.684+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f860014e8c0 msgr2=0x7f8600155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.684+0000 7f8607d31640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f860014e8c0 0x7f8600155ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d530 tx=0x7f85f000d9f0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.685+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 shutdown_connections 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.685+0000 7f8607d31640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f860014e8c0 0x7f8600155ed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.685+0000 7f8607d31640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86000af750 0x7f860014e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.685+0000 7f8607d31640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f85e406c940 0x7f85e406ee00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.685+0000 7f8607d31640 1 --2- 172.21.15.135:0/1399195512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f860006c0a0 0x7f860014cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.685+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 >> 172.21.15.135:0/1399195512 conn(0x7f86000b2290 msgr2=0x7f860006b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.685+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 shutdown_connections 2024-04-17T06:09:41.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.685+0000 7f8607d31640 1 -- 172.21.15.135:0/1399195512 wait complete. 2024-04-17T06:09:41.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-17T06:09:41.689 DEBUG:teuthology.parallel:result is None 2024-04-17T06:09:41.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2024-04-17T06:09:41.700 DEBUG:teuthology.parallel:result is None 2024-04-17T06:09:41.704 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.707+0000 7f9078ff9640 1 -- 172.21.15.135:0/1595346224 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9060005850 con 0x7f907c038c90 2024-04-17T06:09:41.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.707+0000 7f907affd640 1 -- 172.21.15.135:0/1595346224 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f907407bdb0 con 0x7f907c038c90 2024-04-17T06:09:41.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.707+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f176c052850 con 0x7f176c054390 2024-04-17T06:09:41.705 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-17T06:09:41.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.707+0000 7f176affd640 1 -- 172.21.15.135:0/1455361815 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f17740d4db0 con 0x7f176c054390 2024-04-17T06:09:41.706 INFO:teuthology.orchestra.run.smithi135.stdout:55834574857 2024-04-17T06:09:41.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f175806c970 msgr2=0x7f175806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.708 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f175806c970 0x7f175806ee30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1770006fd0 tx=0x7f1770008040 comp rx=0 tx=0).stop 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c054390 msgr2=0x7f176c0367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c054390 0x7f176c0367c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1774063580 tx=0x7f1774069500 comp rx=0 tx=0).stop 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 shutdown_connections 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f176c054390 0x7f176c0367c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f175806c970 0x7f175806ee30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f176c048100 0x7f176c0e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 --2- 172.21.15.135:0/1455361815 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f176c007460 0x7f176c173120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 >> 172.21.15.135:0/1455361815 conn(0x7f176c009fe0 msgr2=0x7f176c0e5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 shutdown_connections 2024-04-17T06:09:41.709 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.710+0000 7f177c4c5640 1 -- 172.21.15.135:0/1455361815 wait complete. 2024-04-17T06:09:41.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 -- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f906806c9c0 msgr2=0x7f906806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f906806c9c0 0x7f906806ee80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9080006fd0 tx=0x7f9080008040 comp rx=0 tx=0).stop 2024-04-17T06:09:41.710 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 -- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c038c90 msgr2=0x7f907c0e3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c038c90 0x7f907c0e3530 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f907403a000 tx=0x7f90740046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 -- 172.21.15.135:0/1595346224 shutdown_connections 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f907c0e1210 0x7f907c0d3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f906806c9c0 0x7f906806ee80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c038c90 0x7f907c0e3530 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 --2- 172.21.15.135:0/1595346224 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f907c036130 0x7f907c0dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 -- 172.21.15.135:0/1595346224 >> 172.21.15.135:0/1595346224 conn(0x7f907c053750 msgr2=0x7f907c03b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 -- 172.21.15.135:0/1595346224 shutdown_connections 2024-04-17T06:09:41.711 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f9078ff9640 1 -- 172.21.15.135:0/1595346224 wait complete. 2024-04-17T06:09:41.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.711+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f685c005580 con 0x7f687c06dd60 2024-04-17T06:09:41.712 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.712+0000 7f6863fff640 1 -- 172.21.15.135:0/824555902 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f686c059af0 con 0x7f687c06dd60 2024-04-17T06:09:41.712 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:41.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.712+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f11000540b0 con 0x7f110000aaa0 2024-04-17T06:09:41.713 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.713+0000 7f10fe7fc640 1 -- 172.21.15.135:0/1672613412 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f10f805cfa0 con 0x7f110000aaa0 2024-04-17T06:09:41.713 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.714+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f686806c8f0 msgr2=0x7f686806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.714+0000 7f6881e14640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f686806c8f0 0x7f686806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f687c06eb90 tx=0x7f6870008040 comp rx=0 tx=0).stop 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.714+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f687c06dd60 msgr2=0x7f687c1fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.714+0000 7f6881e14640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f687c06dd60 0x7f687c1fc170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f686c00d560 tx=0x7f686c00da20 comp rx=0 tx=0).stop 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.715+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 shutdown_connections 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.715+0000 7f6881e14640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f687c0c8830 0x7f687c1f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.715+0000 7f6881e14640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f687c06e140 0x7f687c0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.715+0000 7f6881e14640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f686806c8f0 0x7f686806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.715+0000 7f6881e14640 1 --2- 172.21.15.135:0/824555902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f687c06dd60 0x7f687c1fc170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.715+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 >> 172.21.15.135:0/824555902 conn(0x7f687c0b41a0 msgr2=0x7f687c162010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.715+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 shutdown_connections 2024-04-17T06:09:41.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.715+0000 7f6881e14640 1 -- 172.21.15.135:0/824555902 wait complete. 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f10ec06c9c0 msgr2=0x7f10ec06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f10ec06c9c0 0x7f10ec06ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1100008680 tx=0x7f110806f030 comp rx=0 tx=0).stop 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f110000aaa0 msgr2=0x7f11000cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f110000aaa0 0x7f11000cfb70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c480 tx=0x7f10f800c940 comp rx=0 tx=0).stop 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 shutdown_connections 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11000df420 0x7f11000d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f10ec06c9c0 0x7f10ec06ee80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11000501a0 0x7f11000cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 --2- 172.21.15.135:0/1672613412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f110000aaa0 0x7f11000cfb70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.716 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.716+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 >> 172.21.15.135:0/1672613412 conn(0x7f1100009fb0 msgr2=0x7f11000dd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.717+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 shutdown_connections 2024-04-17T06:09:41.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.717+0000 7f111010d640 1 -- 172.21.15.135:0/1672613412 wait complete. 2024-04-17T06:09:41.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-04-17T06:09:41.724 DEBUG:teuthology.parallel:result is None 2024-04-17T06:09:41.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.728+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f394c071fd0 con 0x7f394c0c5d70 2024-04-17T06:09:41.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.729+0000 7f39427fc640 1 -- 172.21.15.135:0/1438397712 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3948058ec0 con 0x7f394c0c5d70 2024-04-17T06:09:41.727 INFO:teuthology.orchestra.run.smithi135.stdout:55834574858 2024-04-17T06:09:41.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2024-04-17T06:09:41.729 DEBUG:teuthology.parallel:result is None 2024-04-17T06:09:41.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-04-17T06:09:41.730 DEBUG:teuthology.parallel:result is None 2024-04-17T06:09:41.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f393006ca20 msgr2=0x7f393006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f393006ca20 0x7f393006eee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f393c001440 tx=0x7f393c007990 comp rx=0 tx=0).stop 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c0c5d70 msgr2=0x7f394c0b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c0c5d70 0x7f394c0b0770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f394800c8f0 tx=0x7f394800cdb0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 shutdown_connections 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f394c160ba0 0x7f394c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f393006ca20 0x7f393006eee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f394c0c5d70 0x7f394c0b0770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 --2- 172.21.15.135:0/1438397712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f394c06d120 0x7f394c08b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 >> 172.21.15.135:0/1438397712 conn(0x7f394c0b4180 msgr2=0x7f394c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 shutdown_connections 2024-04-17T06:09:41.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.731+0000 7f3953eb3640 1 -- 172.21.15.135:0/1438397712 wait complete. 2024-04-17T06:09:41.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-17T06:09:41.733 DEBUG:teuthology.parallel:result is None 2024-04-17T06:09:41.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-17T06:09:41.741 DEBUG:teuthology.parallel:result is None 2024-04-17T06:09:41.741 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-17T06:09:41.741 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T06:09:41.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.886+0000 7f42d8a2a640 1 Processor -- start 2024-04-17T06:09:41.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.886+0000 7f42d8a2a640 1 -- start start 2024-04-17T06:09:41.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d8a2a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f42d4162190 0x7f42d406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d8a2a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f42d409b100 0x7f42d40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d8a2a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d40d2180 con 0x7f42d40d1cf0 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d8a2a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f42d40ccd00 con 0x7f42d409b100 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d8a2a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f42d406ddc0 con 0x7f42d4162190 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d2d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f42d409b100 0x7f42d40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d2d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f42d409b100 0x7f42d40cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49024/0 (socket says 172.21.15.135:49024) 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d2d9d640 1 -- 172.21.15.135:0/2156003833 learned_addr learned my addr 172.21.15.135:0/2156003833 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d259c640 1 --2- 172.21.15.135:0/2156003833 >> v2:172.21.15.190:3300/0 conn(0x7f42d4162190 0x7f42d406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d2d9d640 1 -- 172.21.15.135:0/2156003833 >> v2:172.21.15.190:3300/0 conn(0x7f42d4162190 msgr2=0x7f42d406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d2d9d640 1 --2- 172.21.15.135:0/2156003833 >> v2:172.21.15.190:3300/0 conn(0x7f42d4162190 0x7f42d406d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d2d9d640 1 -- 172.21.15.135:0/2156003833 >> v1:172.21.15.190:6789/0 conn(0x7f42d40d1cf0 legacy=0x7f42d409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.887+0000 7f42d2d9d640 1 -- 172.21.15.135:0/2156003833 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d406e0a0 con 0x7f42d409b100 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.888+0000 7f42d2d9d640 1 --2- 172.21.15.135:0/2156003833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d40cd7c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42d40739d0 tx=0x7f42cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c566529e6bde584a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.888+0000 7f42d159a640 1 -- 172.21.15.135:0/2156003833 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc034070 con 0x7f42d409b100 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.888+0000 7f42d159a640 1 -- 172.21.15.135:0/2156003833 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42cc030860 con 0x7f42d409b100 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.888+0000 7f42d159a640 1 -- 172.21.15.135:0/2156003833 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc030c30 con 0x7f42d409b100 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.889+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2156003833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 msgr2=0x7f42d40cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.889+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2156003833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d40cd7c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42d40739d0 tx=0x7f42cc02b9a0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.889+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2156003833 shutdown_connections 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.889+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2156003833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d40cd7c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.889+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2156003833 >> v2:172.21.15.190:3300/0 conn(0x7f42d4162190 0x7f42d406d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.889+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2156003833 >> 172.21.15.135:0/2156003833 conn(0x7f42d40b41a0 msgr2=0x7f42d40b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.889+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2156003833 shutdown_connections 2024-04-17T06:09:41.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.889+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2156003833 wait complete. 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.890+0000 7f42d8a2a640 1 Processor -- start 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.890+0000 7f42d8a2a640 1 -- start start 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d8a2a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d41f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d8a2a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42d40d1cf0 0x7f42d40b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d8a2a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4162190 0x7f42d40b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d8a2a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42d4162570 con 0x7f42d409b100 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d8a2a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40cd2a0 con 0x7f42d4162190 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d8a2a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f42d40cdd00 con 0x7f42d40d1cf0 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d41f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d41f3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49026/0 (socket says 172.21.15.135:49026) 2024-04-17T06:09:41.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d259c640 1 -- 172.21.15.135:0/2907821032 learned_addr learned my addr 172.21.15.135:0/2907821032 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d1d9b640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42d40d1cf0 0x7f42d40b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d2d9d640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4162190 0x7f42d40b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d259c640 1 -- 172.21.15.135:0/2907821032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42d40d1cf0 msgr2=0x7f42d40b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d259c640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42d40d1cf0 0x7f42d40b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d259c640 1 -- 172.21.15.135:0/2907821032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4162190 msgr2=0x7f42d40b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d259c640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4162190 0x7f42d40b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.891+0000 7f42d259c640 1 -- 172.21.15.135:0/2907821032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d40b5f10 con 0x7f42d409b100 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.892+0000 7f42d259c640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d41f3b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f42c400efc0 tx=0x7f42c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.892+0000 7f42c2ffd640 1 -- 172.21.15.135:0/2907821032 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4018020 con 0x7f42d409b100 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.892+0000 7f42c2ffd640 1 -- 172.21.15.135:0/2907821032 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c4014480 con 0x7f42d409b100 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.892+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d40b61f0 con 0x7f42d409b100 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.892+0000 7f42c2ffd640 1 -- 172.21.15.135:0/2907821032 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4014850 con 0x7f42d409b100 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.892+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d40b67b0 con 0x7f42d409b100 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.893+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d4074940 con 0x7f42d409b100 2024-04-17T06:09:41.892 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.893+0000 7f42c2ffd640 1 -- 172.21.15.135:0/2907821032 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42c4005d10 con 0x7f42d409b100 2024-04-17T06:09:41.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.894+0000 7f42c2ffd640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f42b406c8f0 0x7f42b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:41.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.894+0000 7f42c2ffd640 1 -- 172.21.15.135:0/2907821032 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f42c408c9b0 con 0x7f42d409b100 2024-04-17T06:09:41.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.894+0000 7f42d1d9b640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f42b406c8f0 0x7f42b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:41.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.894+0000 7f42d1d9b640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f42b406c8f0 0x7f42b406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42d406d620 tx=0x7f42c8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:41.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:41.898+0000 7f42c2ffd640 1 -- 172.21.15.135:0/2907821032 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42c4022070 con 0x7f42d409b100 2024-04-17T06:09:42.069 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.071+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 --> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42d40b6b80 con 0x7f42b406c8f0 2024-04-17T06:09:42.076 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.078+0000 7f42c2ffd640 1 -- 172.21.15.135:0/2907821032 <== mgr.4113 v2:172.21.15.135:6832/3190814219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f42d40b6b80 con 0x7f42b406c8f0 2024-04-17T06:09:42.080 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:42.082 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f42b406c8f0 msgr2=0x7f42b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f42b406c8f0 0x7f42b406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f42d406d620 tx=0x7f42c8009040 comp rx=0 tx=0).stop 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 msgr2=0x7f42d41f3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d41f3b90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f42c400efc0 tx=0x7f42c400c490 comp rx=0 tx=0).stop 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 shutdown_connections 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f42d4162190 0x7f42d40b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f42d40d1cf0 0x7f42d40b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f42b406c8f0 0x7f42b406edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 --2- 172.21.15.135:0/2907821032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42d409b100 0x7f42d41f3b90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 >> 172.21.15.135:0/2907821032 conn(0x7f42d40b41a0 msgr2=0x7f42d406ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:42.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 shutdown_connections 2024-04-17T06:09:42.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.082+0000 7f42d8a2a640 1 -- 172.21.15.135:0/2907821032 wait complete. 2024-04-17T06:09:42.091 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-17T06:09:41.942268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002585"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012356+0000","last_change":"2024-04-17T06:09:19.632439+0000","last_active":"2024-04-17T06:09:26.012356+0000","last_peered":"2024-04-17T06:09:26.012356+0000","last_clean":"2024-04-17T06:09:26.012356+0000","last_became_active":"2024-04-17T06:09:19.632170+0000","last_became_peered":"2024-04-17T06:09:19.632170+0000","last_unstale":"2024-04-17T06:09:26.012356+0000","last_undegraded":"2024-04-17T06:09:26.012356+0000","last_fullsized":"2024-04-17T06:09:26.012356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:58.547080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011716+0000","last_change":"2024-04-17T06:09:20.647290+0000","last_active":"2024-04-17T06:09:26.011716+0000","last_peered":"2024-04-17T06:09:26.011716+0000","last_clean":"2024-04-17T06:09:26.011716+0000","last_became_active":"2024-04-17T06:09:20.646950+0000","last_became_peered":"2024-04-17T06:09:20.646950+0000","last_unstale":"2024-04-17T06:09:26.011716+0000","last_undegraded":"2024-04-17T06:09:26.011716+0000","last_fullsized":"2024-04-17T06:09:26.011716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011791+0000","last_change":"2024-04-17T06:09:24.992961+0000","last_active":"2024-04-17T06:09:26.011791+0000","last_peered":"2024-04-17T06:09:26.011791+0000","last_clean":"2024-04-17T06:09:26.011791+0000","last_became_active":"2024-04-17T06:09:24.992540+0000","last_became_peered":"2024-04-17T06:09:24.992540+0000","last_unstale":"2024-04-17T06:09:26.011791+0000","last_undegraded":"2024-04-17T06:09:26.011791+0000","last_fullsized":"2024-04-17T06:09:26.011791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:42.943499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011930+0000","last_change":"2024-04-17T06:09:19.632406+0000","last_active":"2024-04-17T06:09:26.011930+0000","last_peered":"2024-04-17T06:09:26.011930+0000","last_clean":"2024-04-17T06:09:26.011930+0000","last_became_active":"2024-04-17T06:09:19.631990+0000","last_became_peered":"2024-04-17T06:09:19.631990+0000","last_unstale":"2024-04-17T06:09:26.011930+0000","last_undegraded":"2024-04-17T06:09:26.011930+0000","last_fullsized":"2024-04-17T06:09:26.011930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:35.753743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012065+0000","last_change":"2024-04-17T06:09:20.631799+0000","last_active":"2024-04-17T06:09:26.012065+0000","last_peered":"2024-04-17T06:09:26.012065+0000","last_clean":"2024-04-17T06:09:26.012065+0000","last_became_active":"2024-04-17T06:09:20.631551+0000","last_became_peered":"2024-04-17T06:09:20.631551+0000","last_unstale":"2024-04-17T06:09:26.012065+0000","last_undegraded":"2024-04-17T06:09:26.012065+0000","last_fullsized":"2024-04-17T06:09:26.012065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011413+0000","last_change":"2024-04-17T06:09:24.681573+0000","last_active":"2024-04-17T06:09:26.011413+0000","last_peered":"2024-04-17T06:09:26.011413+0000","last_clean":"2024-04-17T06:09:26.011413+0000","last_became_active":"2024-04-17T06:09:24.681194+0000","last_became_peered":"2024-04-17T06:09:24.681194+0000","last_unstale":"2024-04-17T06:09:26.011413+0000","last_undegraded":"2024-04-17T06:09:26.011413+0000","last_fullsized":"2024-04-17T06:09:26.011413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010848+0000","last_change":"2024-04-17T06:09:19.626563+0000","last_active":"2024-04-17T06:09:26.010848+0000","last_peered":"2024-04-17T06:09:26.010848+0000","last_clean":"2024-04-17T06:09:26.010848+0000","last_became_active":"2024-04-17T06:09:19.626169+0000","last_became_peered":"2024-04-17T06:09:19.626169+0000","last_unstale":"2024-04-17T06:09:26.010848+0000","last_undegraded":"2024-04-17T06:09:26.010848+0000","last_fullsized":"2024-04-17T06:09:26.010848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:07.389883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660478+0000","last_change":"2024-04-17T06:09:20.629675+0000","last_active":"2024-04-17T06:09:25.660478+0000","last_peered":"2024-04-17T06:09:25.660478+0000","last_clean":"2024-04-17T06:09:25.660478+0000","last_became_active":"2024-04-17T06:09:20.629335+0000","last_became_peered":"2024-04-17T06:09:20.629335+0000","last_unstale":"2024-04-17T06:09:25.660478+0000","last_undegraded":"2024-04-17T06:09:25.660478+0000","last_fullsized":"2024-04-17T06:09:25.660478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010974+0000","last_change":"2024-04-17T06:09:24.681484+0000","last_active":"2024-04-17T06:09:26.010974+0000","last_peered":"2024-04-17T06:09:26.010974+0000","last_clean":"2024-04-17T06:09:26.010974+0000","last_became_active":"2024-04-17T06:09:24.681074+0000","last_became_peered":"2024-04-17T06:09:24.681074+0000","last_unstale":"2024-04-17T06:09:26.010974+0000","last_undegraded":"2024-04-17T06:09:26.010974+0000","last_fullsized":"2024-04-17T06:09:26.010974+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:21.508074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011016+0000","last_change":"2024-04-17T06:09:19.629666+0000","last_active":"2024-04-17T06:09:26.011016+0000","last_peered":"2024-04-17T06:09:26.011016+0000","last_clean":"2024-04-17T06:09:26.011016+0000","last_became_active":"2024-04-17T06:09:19.629414+0000","last_became_peered":"2024-04-17T06:09:19.629414+0000","last_unstale":"2024-04-17T06:09:26.011016+0000","last_undegraded":"2024-04-17T06:09:26.011016+0000","last_fullsized":"2024-04-17T06:09:26.011016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660727+0000","last_change":"2024-04-17T06:09:20.629643+0000","last_active":"2024-04-17T06:09:25.660727+0000","last_peered":"2024-04-17T06:09:25.660727+0000","last_clean":"2024-04-17T06:09:25.660727+0000","last_became_active":"2024-04-17T06:09:20.629349+0000","last_became_peered":"2024-04-17T06:09:20.629349+0000","last_unstale":"2024-04-17T06:09:25.660727+0000","last_undegraded":"2024-04-17T06:09:25.660727+0000","last_fullsized":"2024-04-17T06:09:25.660727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:55.368933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011234+0000","last_change":"2024-04-17T06:09:24.673766+0000","last_active":"2024-04-17T06:09:26.011234+0000","last_peered":"2024-04-17T06:09:26.011234+0000","last_clean":"2024-04-17T06:09:26.011234+0000","last_became_active":"2024-04-17T06:09:24.673384+0000","last_became_peered":"2024-04-17T06:09:24.673384+0000","last_unstale":"2024-04-17T06:09:26.011234+0000","last_undegraded":"2024-04-17T06:09:26.011234+0000","last_fullsized":"2024-04-17T06:09:26.011234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:40.236016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659389+0000","last_change":"2024-04-17T06:09:20.643805+0000","last_active":"2024-04-17T06:09:25.659389+0000","last_peered":"2024-04-17T06:09:25.659389+0000","last_clean":"2024-04-17T06:09:25.659389+0000","last_became_active":"2024-04-17T06:09:20.643199+0000","last_became_peered":"2024-04-17T06:09:20.643199+0000","last_unstale":"2024-04-17T06:09:25.659389+0000","last_undegraded":"2024-04-17T06:09:25.659389+0000","last_fullsized":"2024-04-17T06:09:25.659389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:07.365672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011214+0000","last_change":"2024-04-17T06:09:19.626503+0000","last_active":"2024-04-17T06:09:26.011214+0000","last_peered":"2024-04-17T06:09:26.011214+0000","last_clean":"2024-04-17T06:09:26.011214+0000","last_became_active":"2024-04-17T06:09:19.625556+0000","last_became_peered":"2024-04-17T06:09:19.625556+0000","last_unstale":"2024-04-17T06:09:26.011214+0000","last_undegraded":"2024-04-17T06:09:26.011214+0000","last_fullsized":"2024-04-17T06:09:26.011214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010558+0000","last_change":"2024-04-17T06:09:24.680158+0000","last_active":"2024-04-17T06:09:26.010558+0000","last_peered":"2024-04-17T06:09:26.010558+0000","last_clean":"2024-04-17T06:09:26.010558+0000","last_became_active":"2024-04-17T06:09:24.679694+0000","last_became_peered":"2024-04-17T06:09:24.679694+0000","last_unstale":"2024-04-17T06:09:26.010558+0000","last_undegraded":"2024-04-17T06:09:26.010558+0000","last_fullsized":"2024-04-17T06:09:26.010558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.282988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011354+0000","last_change":"2024-04-17T06:09:20.638610+0000","last_active":"2024-04-17T06:09:26.011354+0000","last_peered":"2024-04-17T06:09:26.011354+0000","last_clean":"2024-04-17T06:09:26.011354+0000","last_became_active":"2024-04-17T06:09:20.638275+0000","last_became_peered":"2024-04-17T06:09:20.638275+0000","last_unstale":"2024-04-17T06:09:26.011354+0000","last_undegraded":"2024-04-17T06:09:26.011354+0000","last_fullsized":"2024-04-17T06:09:26.011354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011423+0000","last_change":"2024-04-17T06:09:19.626821+0000","last_active":"2024-04-17T06:09:26.011423+0000","last_peered":"2024-04-17T06:09:26.011423+0000","last_clean":"2024-04-17T06:09:26.011423+0000","last_became_active":"2024-04-17T06:09:19.626550+0000","last_became_peered":"2024-04-17T06:09:19.626550+0000","last_unstale":"2024-04-17T06:09:26.011423+0000","last_undegraded":"2024-04-17T06:09:26.011423+0000","last_fullsized":"2024-04-17T06:09:26.011423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010388+0000","last_change":"2024-04-17T06:09:24.680040+0000","last_active":"2024-04-17T06:09:26.010388+0000","last_peered":"2024-04-17T06:09:26.010388+0000","last_clean":"2024-04-17T06:09:26.010388+0000","last_became_active":"2024-04-17T06:09:24.679567+0000","last_became_peered":"2024-04-17T06:09:24.679567+0000","last_unstale":"2024-04-17T06:09:26.010388+0000","last_undegraded":"2024-04-17T06:09:26.010388+0000","last_fullsized":"2024-04-17T06:09:26.010388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:21.549944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010400+0000","last_change":"2024-04-17T06:09:20.645617+0000","last_active":"2024-04-17T06:09:26.010400+0000","last_peered":"2024-04-17T06:09:26.010400+0000","last_clean":"2024-04-17T06:09:26.010400+0000","last_became_active":"2024-04-17T06:09:20.645205+0000","last_became_peered":"2024-04-17T06:09:20.645205+0000","last_unstale":"2024-04-17T06:09:26.010400+0000","last_undegraded":"2024-04-17T06:09:26.010400+0000","last_fullsized":"2024-04-17T06:09:26.010400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010623+0000","last_change":"2024-04-17T06:09:19.623695+0000","last_active":"2024-04-17T06:09:26.010623+0000","last_peered":"2024-04-17T06:09:26.010623+0000","last_clean":"2024-04-17T06:09:26.010623+0000","last_became_active":"2024-04-17T06:09:19.623442+0000","last_became_peered":"2024-04-17T06:09:19.623442+0000","last_unstale":"2024-04-17T06:09:26.010623+0000","last_undegraded":"2024-04-17T06:09:26.010623+0000","last_fullsized":"2024-04-17T06:09:26.010623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:07.888042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010887+0000","last_change":"2024-04-17T06:09:24.990476+0000","last_active":"2024-04-17T06:09:26.010887+0000","last_peered":"2024-04-17T06:09:26.010887+0000","last_clean":"2024-04-17T06:09:26.010887+0000","last_became_active":"2024-04-17T06:09:24.990092+0000","last_became_peered":"2024-04-17T06:09:24.990092+0000","last_unstale":"2024-04-17T06:09:26.010887+0000","last_undegraded":"2024-04-17T06:09:26.010887+0000","last_fullsized":"2024-04-17T06:09:26.010887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010691+0000","last_change":"2024-04-17T06:09:20.633239+0000","last_active":"2024-04-17T06:09:26.010691+0000","last_peered":"2024-04-17T06:09:26.010691+0000","last_clean":"2024-04-17T06:09:26.010691+0000","last_became_active":"2024-04-17T06:09:20.632946+0000","last_became_peered":"2024-04-17T06:09:20.632946+0000","last_unstale":"2024-04-17T06:09:26.010691+0000","last_undegraded":"2024-04-17T06:09:26.010691+0000","last_fullsized":"2024-04-17T06:09:26.010691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:01.439932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010236+0000","last_change":"2024-04-17T06:09:19.632434+0000","last_active":"2024-04-17T06:09:26.010236+0000","last_peered":"2024-04-17T06:09:26.010236+0000","last_clean":"2024-04-17T06:09:26.010236+0000","last_became_active":"2024-04-17T06:09:19.632174+0000","last_became_peered":"2024-04-17T06:09:19.632174+0000","last_unstale":"2024-04-17T06:09:26.010236+0000","last_undegraded":"2024-04-17T06:09:26.010236+0000","last_fullsized":"2024-04-17T06:09:26.010236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:06.724008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010716+0000","last_change":"2024-04-17T06:09:24.680395+0000","last_active":"2024-04-17T06:09:26.010716+0000","last_peered":"2024-04-17T06:09:26.010716+0000","last_clean":"2024-04-17T06:09:26.010716+0000","last_became_active":"2024-04-17T06:09:24.679895+0000","last_became_peered":"2024-04-17T06:09:24.679895+0000","last_unstale":"2024-04-17T06:09:26.010716+0000","last_undegraded":"2024-04-17T06:09:26.010716+0000","last_fullsized":"2024-04-17T06:09:26.010716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:14.465883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012593+0000","last_change":"2024-04-17T06:09:20.638642+0000","last_active":"2024-04-17T06:09:26.012593+0000","last_peered":"2024-04-17T06:09:26.012593+0000","last_clean":"2024-04-17T06:09:26.012593+0000","last_became_active":"2024-04-17T06:09:20.638351+0000","last_became_peered":"2024-04-17T06:09:20.638351+0000","last_unstale":"2024-04-17T06:09:26.012593+0000","last_undegraded":"2024-04-17T06:09:26.012593+0000","last_fullsized":"2024-04-17T06:09:26.012593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660496+0000","last_change":"2024-04-17T06:09:19.622032+0000","last_active":"2024-04-17T06:09:25.660496+0000","last_peered":"2024-04-17T06:09:25.660496+0000","last_clean":"2024-04-17T06:09:25.660496+0000","last_became_active":"2024-04-17T06:09:19.621765+0000","last_became_peered":"2024-04-17T06:09:19.621765+0000","last_unstale":"2024-04-17T06:09:25.660496+0000","last_undegraded":"2024-04-17T06:09:25.660496+0000","last_fullsized":"2024-04-17T06:09:25.660496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010414+0000","last_change":"2024-04-17T06:09:24.990792+0000","last_active":"2024-04-17T06:09:26.010414+0000","last_peered":"2024-04-17T06:09:26.010414+0000","last_clean":"2024-04-17T06:09:26.010414+0000","last_became_active":"2024-04-17T06:09:24.990371+0000","last_became_peered":"2024-04-17T06:09:24.990371+0000","last_unstale":"2024-04-17T06:09:26.010414+0000","last_undegraded":"2024-04-17T06:09:26.010414+0000","last_fullsized":"2024-04-17T06:09:26.010414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:26.189133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011005+0000","last_change":"2024-04-17T06:09:20.648546+0000","last_active":"2024-04-17T06:09:26.011005+0000","last_peered":"2024-04-17T06:09:26.011005+0000","last_clean":"2024-04-17T06:09:26.011005+0000","last_became_active":"2024-04-17T06:09:20.647853+0000","last_became_peered":"2024-04-17T06:09:20.647853+0000","last_unstale":"2024-04-17T06:09:26.011005+0000","last_undegraded":"2024-04-17T06:09:26.011005+0000","last_fullsized":"2024-04-17T06:09:26.011005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011025+0000","last_change":"2024-04-17T06:09:19.627150+0000","last_active":"2024-04-17T06:09:26.011025+0000","last_peered":"2024-04-17T06:09:26.011025+0000","last_clean":"2024-04-17T06:09:26.011025+0000","last_became_active":"2024-04-17T06:09:19.626875+0000","last_became_peered":"2024-04-17T06:09:19.626875+0000","last_unstale":"2024-04-17T06:09:26.011025+0000","last_undegraded":"2024-04-17T06:09:26.011025+0000","last_fullsized":"2024-04-17T06:09:26.011025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010576+0000","last_change":"2024-04-17T06:09:24.992298+0000","last_active":"2024-04-17T06:09:26.010576+0000","last_peered":"2024-04-17T06:09:26.010576+0000","last_clean":"2024-04-17T06:09:26.010576+0000","last_became_active":"2024-04-17T06:09:24.991871+0000","last_became_peered":"2024-04-17T06:09:24.991871+0000","last_unstale":"2024-04-17T06:09:26.010576+0000","last_undegraded":"2024-04-17T06:09:26.010576+0000","last_fullsized":"2024-04-17T06:09:26.010576+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:43.835421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011016+0000","last_change":"2024-04-17T06:09:20.639206+0000","last_active":"2024-04-17T06:09:26.011016+0000","last_peered":"2024-04-17T06:09:26.011016+0000","last_clean":"2024-04-17T06:09:26.011016+0000","last_became_active":"2024-04-17T06:09:20.638954+0000","last_became_peered":"2024-04-17T06:09:20.638954+0000","last_unstale":"2024-04-17T06:09:26.011016+0000","last_undegraded":"2024-04-17T06:09:26.011016+0000","last_fullsized":"2024-04-17T06:09:26.011016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011535+0000","last_change":"2024-04-17T06:09:19.632347+0000","last_active":"2024-04-17T06:09:26.011535+0000","last_peered":"2024-04-17T06:09:26.011535+0000","last_clean":"2024-04-17T06:09:26.011535+0000","last_became_active":"2024-04-17T06:09:19.632117+0000","last_became_peered":"2024-04-17T06:09:19.632117+0000","last_unstale":"2024-04-17T06:09:26.011535+0000","last_undegraded":"2024-04-17T06:09:26.011535+0000","last_fullsized":"2024-04-17T06:09:26.011535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:07.888042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011384+0000","last_change":"2024-04-17T06:09:24.989732+0000","last_active":"2024-04-17T06:09:26.011384+0000","last_peered":"2024-04-17T06:09:26.011384+0000","last_clean":"2024-04-17T06:09:26.011384+0000","last_became_active":"2024-04-17T06:09:24.989320+0000","last_became_peered":"2024-04-17T06:09:24.989320+0000","last_unstale":"2024-04-17T06:09:26.011384+0000","last_undegraded":"2024-04-17T06:09:26.011384+0000","last_fullsized":"2024-04-17T06:09:26.011384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660737+0000","last_change":"2024-04-17T06:09:20.652167+0000","last_active":"2024-04-17T06:09:25.660737+0000","last_peered":"2024-04-17T06:09:25.660737+0000","last_clean":"2024-04-17T06:09:25.660737+0000","last_became_active":"2024-04-17T06:09:20.651908+0000","last_became_peered":"2024-04-17T06:09:20.651908+0000","last_unstale":"2024-04-17T06:09:25.660737+0000","last_undegraded":"2024-04-17T06:09:25.660737+0000","last_fullsized":"2024-04-17T06:09:25.660737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:58.610924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011096+0000","last_change":"2024-04-17T06:09:19.631750+0000","last_active":"2024-04-17T06:09:26.011096+0000","last_peered":"2024-04-17T06:09:26.011096+0000","last_clean":"2024-04-17T06:09:26.011096+0000","last_became_active":"2024-04-17T06:09:19.631463+0000","last_became_peered":"2024-04-17T06:09:19.631463+0000","last_unstale":"2024-04-17T06:09:26.011096+0000","last_undegraded":"2024-04-17T06:09:26.011096+0000","last_fullsized":"2024-04-17T06:09:26.011096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:07.389883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011105+0000","last_change":"2024-04-17T06:09:24.680787+0000","last_active":"2024-04-17T06:09:26.011105+0000","last_peered":"2024-04-17T06:09:26.011105+0000","last_clean":"2024-04-17T06:09:26.011105+0000","last_became_active":"2024-04-17T06:09:24.680279+0000","last_became_peered":"2024-04-17T06:09:24.680279+0000","last_unstale":"2024-04-17T06:09:26.011105+0000","last_undegraded":"2024-04-17T06:09:26.011105+0000","last_fullsized":"2024-04-17T06:09:26.011105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:25.702449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011262+0000","last_change":"2024-04-17T06:09:20.632789+0000","last_active":"2024-04-17T06:09:26.011262+0000","last_peered":"2024-04-17T06:09:26.011262+0000","last_clean":"2024-04-17T06:09:26.011262+0000","last_became_active":"2024-04-17T06:09:20.632199+0000","last_became_peered":"2024-04-17T06:09:20.632199+0000","last_unstale":"2024-04-17T06:09:26.011262+0000","last_undegraded":"2024-04-17T06:09:26.011262+0000","last_fullsized":"2024-04-17T06:09:26.011262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:08.393904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660530+0000","last_change":"2024-04-17T06:09:19.624615+0000","last_active":"2024-04-17T06:09:25.660530+0000","last_peered":"2024-04-17T06:09:25.660530+0000","last_clean":"2024-04-17T06:09:25.660530+0000","last_became_active":"2024-04-17T06:09:19.624333+0000","last_became_peered":"2024-04-17T06:09:19.624333+0000","last_unstale":"2024-04-17T06:09:25.660530+0000","last_undegraded":"2024-04-17T06:09:25.660530+0000","last_fullsized":"2024-04-17T06:09:25.660530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011059+0000","last_change":"2024-04-17T06:09:24.675868+0000","last_active":"2024-04-17T06:09:26.011059+0000","last_peered":"2024-04-17T06:09:26.011059+0000","last_clean":"2024-04-17T06:09:26.011059+0000","last_became_active":"2024-04-17T06:09:24.675269+0000","last_became_peered":"2024-04-17T06:09:24.675269+0000","last_unstale":"2024-04-17T06:09:26.011059+0000","last_undegraded":"2024-04-17T06:09:26.011059+0000","last_fullsized":"2024-04-17T06:09:26.011059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:02.650317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010687+0000","last_change":"2024-04-17T06:09:20.638626+0000","last_active":"2024-04-17T06:09:26.010687+0000","last_peered":"2024-04-17T06:09:26.010687+0000","last_clean":"2024-04-17T06:09:26.010687+0000","last_became_active":"2024-04-17T06:09:20.638339+0000","last_became_peered":"2024-04-17T06:09:20.638339+0000","last_unstale":"2024-04-17T06:09:26.010687+0000","last_undegraded":"2024-04-17T06:09:26.010687+0000","last_fullsized":"2024-04-17T06:09:26.010687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661071+0000","last_change":"2024-04-17T06:09:19.621292+0000","last_active":"2024-04-17T06:09:25.661071+0000","last_peered":"2024-04-17T06:09:25.661071+0000","last_clean":"2024-04-17T06:09:25.661071+0000","last_became_active":"2024-04-17T06:09:19.620827+0000","last_became_peered":"2024-04-17T06:09:19.620827+0000","last_unstale":"2024-04-17T06:09:25.661071+0000","last_undegraded":"2024-04-17T06:09:25.661071+0000","last_fullsized":"2024-04-17T06:09:25.661071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011064+0000","last_change":"2024-04-17T06:09:24.682574+0000","last_active":"2024-04-17T06:09:26.011064+0000","last_peered":"2024-04-17T06:09:26.011064+0000","last_clean":"2024-04-17T06:09:26.011064+0000","last_became_active":"2024-04-17T06:09:24.680550+0000","last_became_peered":"2024-04-17T06:09:24.680550+0000","last_unstale":"2024-04-17T06:09:26.011064+0000","last_undegraded":"2024-04-17T06:09:26.011064+0000","last_fullsized":"2024-04-17T06:09:26.011064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:20.183387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010301+0000","last_change":"2024-04-17T06:09:20.630308+0000","last_active":"2024-04-17T06:09:26.010301+0000","last_peered":"2024-04-17T06:09:26.010301+0000","last_clean":"2024-04-17T06:09:26.010301+0000","last_became_active":"2024-04-17T06:09:20.630052+0000","last_became_peered":"2024-04-17T06:09:20.630052+0000","last_unstale":"2024-04-17T06:09:26.010301+0000","last_undegraded":"2024-04-17T06:09:26.010301+0000","last_fullsized":"2024-04-17T06:09:26.010301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:53.394999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010349+0000","last_change":"2024-04-17T06:09:19.624643+0000","last_active":"2024-04-17T06:09:26.010349+0000","last_peered":"2024-04-17T06:09:26.010349+0000","last_clean":"2024-04-17T06:09:26.010349+0000","last_became_active":"2024-04-17T06:09:19.624270+0000","last_became_peered":"2024-04-17T06:09:19.624270+0000","last_unstale":"2024-04-17T06:09:26.010349+0000","last_undegraded":"2024-04-17T06:09:26.010349+0000","last_fullsized":"2024-04-17T06:09:26.010349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012635+0000","last_change":"2024-04-17T06:09:24.989917+0000","last_active":"2024-04-17T06:09:26.012635+0000","last_peered":"2024-04-17T06:09:26.012635+0000","last_clean":"2024-04-17T06:09:26.012635+0000","last_became_active":"2024-04-17T06:09:24.989308+0000","last_became_peered":"2024-04-17T06:09:24.989308+0000","last_unstale":"2024-04-17T06:09:26.012635+0000","last_undegraded":"2024-04-17T06:09:26.012635+0000","last_fullsized":"2024-04-17T06:09:26.012635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:14:19.804372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010303+0000","last_change":"2024-04-17T06:09:20.656161+0000","last_active":"2024-04-17T06:09:26.010303+0000","last_peered":"2024-04-17T06:09:26.010303+0000","last_clean":"2024-04-17T06:09:26.010303+0000","last_became_active":"2024-04-17T06:09:20.655897+0000","last_became_peered":"2024-04-17T06:09:20.655897+0000","last_unstale":"2024-04-17T06:09:26.010303+0000","last_undegraded":"2024-04-17T06:09:26.010303+0000","last_fullsized":"2024-04-17T06:09:26.010303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660641+0000","last_change":"2024-04-17T06:09:19.621223+0000","last_active":"2024-04-17T06:09:25.660641+0000","last_peered":"2024-04-17T06:09:25.660641+0000","last_clean":"2024-04-17T06:09:25.660641+0000","last_became_active":"2024-04-17T06:09:19.620409+0000","last_became_peered":"2024-04-17T06:09:19.620409+0000","last_unstale":"2024-04-17T06:09:25.660641+0000","last_undegraded":"2024-04-17T06:09:25.660641+0000","last_fullsized":"2024-04-17T06:09:25.660641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010841+0000","last_change":"2024-04-17T06:09:09.067035+0000","last_active":"2024-04-17T06:09:26.010841+0000","last_peered":"2024-04-17T06:09:26.010841+0000","last_clean":"2024-04-17T06:09:26.010841+0000","last_became_active":"2024-04-17T06:09:09.066130+0000","last_became_peered":"2024-04-17T06:09:09.066130+0000","last_unstale":"2024-04-17T06:09:26.010841+0000","last_undegraded":"2024-04-17T06:09:26.010841+0000","last_fullsized":"2024-04-17T06:09:26.010841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_clean_scrub_stamp":"2024-04-17T06:09:07.958875+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:03.277836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659861+0000","last_change":"2024-04-17T06:09:24.992008+0000","last_active":"2024-04-17T06:09:25.659861+0000","last_peered":"2024-04-17T06:09:25.659861+0000","last_clean":"2024-04-17T06:09:25.659861+0000","last_became_active":"2024-04-17T06:09:24.991448+0000","last_became_peered":"2024-04-17T06:09:24.991448+0000","last_unstale":"2024-04-17T06:09:25.659861+0000","last_undegraded":"2024-04-17T06:09:25.659861+0000","last_fullsized":"2024-04-17T06:09:25.659861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:15.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010640+0000","last_change":"2024-04-17T06:09:20.646916+0000","last_active":"2024-04-17T06:09:26.010640+0000","last_peered":"2024-04-17T06:09:26.010640+0000","last_clean":"2024-04-17T06:09:26.010640+0000","last_became_active":"2024-04-17T06:09:20.646069+0000","last_became_peered":"2024-04-17T06:09:20.646069+0000","last_unstale":"2024-04-17T06:09:26.010640+0000","last_undegraded":"2024-04-17T06:09:26.010640+0000","last_fullsized":"2024-04-17T06:09:26.010640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:11.442995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661446+0000","last_change":"2024-04-17T06:09:19.626825+0000","last_active":"2024-04-17T06:09:25.661446+0000","last_peered":"2024-04-17T06:09:25.661446+0000","last_clean":"2024-04-17T06:09:25.661446+0000","last_became_active":"2024-04-17T06:09:19.626549+0000","last_became_peered":"2024-04-17T06:09:19.626549+0000","last_unstale":"2024-04-17T06:09:25.661446+0000","last_undegraded":"2024-04-17T06:09:25.661446+0000","last_fullsized":"2024-04-17T06:09:25.661446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659329+0000","last_change":"2024-04-17T06:09:24.990665+0000","last_active":"2024-04-17T06:09:25.659329+0000","last_peered":"2024-04-17T06:09:25.659329+0000","last_clean":"2024-04-17T06:09:25.659329+0000","last_became_active":"2024-04-17T06:09:24.990262+0000","last_became_peered":"2024-04-17T06:09:24.990262+0000","last_unstale":"2024-04-17T06:09:25.659329+0000","last_undegraded":"2024-04-17T06:09:25.659329+0000","last_fullsized":"2024-04-17T06:09:25.659329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:21.508074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660652+0000","last_change":"2024-04-17T06:09:20.650264+0000","last_active":"2024-04-17T06:09:25.660652+0000","last_peered":"2024-04-17T06:09:25.660652+0000","last_clean":"2024-04-17T06:09:25.660652+0000","last_became_active":"2024-04-17T06:09:20.649792+0000","last_became_peered":"2024-04-17T06:09:20.649792+0000","last_unstale":"2024-04-17T06:09:25.660652+0000","last_undegraded":"2024-04-17T06:09:25.660652+0000","last_fullsized":"2024-04-17T06:09:25.660652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:36.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011681+0000","last_change":"2024-04-17T06:09:19.625656+0000","last_active":"2024-04-17T06:09:26.011681+0000","last_peered":"2024-04-17T06:09:26.011681+0000","last_clean":"2024-04-17T06:09:26.011681+0000","last_became_active":"2024-04-17T06:09:19.625408+0000","last_became_peered":"2024-04-17T06:09:19.625408+0000","last_unstale":"2024-04-17T06:09:26.011681+0000","last_undegraded":"2024-04-17T06:09:26.011681+0000","last_fullsized":"2024-04-17T06:09:26.011681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011611+0000","last_change":"2024-04-17T06:09:24.684050+0000","last_active":"2024-04-17T06:09:26.011611+0000","last_peered":"2024-04-17T06:09:26.011611+0000","last_clean":"2024-04-17T06:09:26.011611+0000","last_became_active":"2024-04-17T06:09:24.683688+0000","last_became_peered":"2024-04-17T06:09:24.683688+0000","last_unstale":"2024-04-17T06:09:26.011611+0000","last_undegraded":"2024-04-17T06:09:26.011611+0000","last_fullsized":"2024-04-17T06:09:26.011611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:02.650317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011613+0000","last_change":"2024-04-17T06:09:20.645936+0000","last_active":"2024-04-17T06:09:26.011613+0000","last_peered":"2024-04-17T06:09:26.011613+0000","last_clean":"2024-04-17T06:09:26.011613+0000","last_became_active":"2024-04-17T06:09:20.645453+0000","last_became_peered":"2024-04-17T06:09:20.645453+0000","last_unstale":"2024-04-17T06:09:26.011613+0000","last_undegraded":"2024-04-17T06:09:26.011613+0000","last_fullsized":"2024-04-17T06:09:26.011613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:21.663057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661396+0000","last_change":"2024-04-17T06:09:19.621217+0000","last_active":"2024-04-17T06:09:25.661396+0000","last_peered":"2024-04-17T06:09:25.661396+0000","last_clean":"2024-04-17T06:09:25.661396+0000","last_became_active":"2024-04-17T06:09:19.620846+0000","last_became_peered":"2024-04-17T06:09:19.620846+0000","last_unstale":"2024-04-17T06:09:25.661396+0000","last_undegraded":"2024-04-17T06:09:25.661396+0000","last_fullsized":"2024-04-17T06:09:25.661396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659685+0000","last_change":"2024-04-17T06:09:24.991134+0000","last_active":"2024-04-17T06:09:25.659685+0000","last_peered":"2024-04-17T06:09:25.659685+0000","last_clean":"2024-04-17T06:09:25.659685+0000","last_became_active":"2024-04-17T06:09:24.990742+0000","last_became_peered":"2024-04-17T06:09:24.990742+0000","last_unstale":"2024-04-17T06:09:25.659685+0000","last_undegraded":"2024-04-17T06:09:25.659685+0000","last_fullsized":"2024-04-17T06:09:25.659685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:54.153325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011276+0000","last_change":"2024-04-17T06:09:20.645660+0000","last_active":"2024-04-17T06:09:26.011276+0000","last_peered":"2024-04-17T06:09:26.011276+0000","last_clean":"2024-04-17T06:09:26.011276+0000","last_became_active":"2024-04-17T06:09:20.645370+0000","last_became_peered":"2024-04-17T06:09:20.645370+0000","last_unstale":"2024-04-17T06:09:26.011276+0000","last_undegraded":"2024-04-17T06:09:26.011276+0000","last_fullsized":"2024-04-17T06:09:26.011276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:36.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011721+0000","last_change":"2024-04-17T06:09:19.633064+0000","last_active":"2024-04-17T06:09:26.011721+0000","last_peered":"2024-04-17T06:09:26.011721+0000","last_clean":"2024-04-17T06:09:26.011721+0000","last_became_active":"2024-04-17T06:09:19.632759+0000","last_became_peered":"2024-04-17T06:09:19.632759+0000","last_unstale":"2024-04-17T06:09:26.011721+0000","last_undegraded":"2024-04-17T06:09:26.011721+0000","last_fullsized":"2024-04-17T06:09:26.011721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:58.547080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010213+0000","last_change":"2024-04-17T06:09:24.676408+0000","last_active":"2024-04-17T06:09:26.010213+0000","last_peered":"2024-04-17T06:09:26.010213+0000","last_clean":"2024-04-17T06:09:26.010213+0000","last_became_active":"2024-04-17T06:09:24.675996+0000","last_became_peered":"2024-04-17T06:09:24.675996+0000","last_unstale":"2024-04-17T06:09:26.010213+0000","last_undegraded":"2024-04-17T06:09:26.010213+0000","last_fullsized":"2024-04-17T06:09:26.010213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:14.169938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010896+0000","last_change":"2024-04-17T06:09:20.645388+0000","last_active":"2024-04-17T06:09:26.010896+0000","last_peered":"2024-04-17T06:09:26.010896+0000","last_clean":"2024-04-17T06:09:26.010896+0000","last_became_active":"2024-04-17T06:09:20.645091+0000","last_became_peered":"2024-04-17T06:09:20.645091+0000","last_unstale":"2024-04-17T06:09:26.010896+0000","last_undegraded":"2024-04-17T06:09:26.010896+0000","last_fullsized":"2024-04-17T06:09:26.010896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660949+0000","last_change":"2024-04-17T06:09:19.624671+0000","last_active":"2024-04-17T06:09:25.660949+0000","last_peered":"2024-04-17T06:09:25.660949+0000","last_clean":"2024-04-17T06:09:25.660949+0000","last_became_active":"2024-04-17T06:09:19.624360+0000","last_became_peered":"2024-04-17T06:09:19.624360+0000","last_unstale":"2024-04-17T06:09:25.660949+0000","last_undegraded":"2024-04-17T06:09:25.660949+0000","last_fullsized":"2024-04-17T06:09:25.660949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:35.753743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012475+0000","last_change":"2024-04-17T06:09:24.677224+0000","last_active":"2024-04-17T06:09:26.012475+0000","last_peered":"2024-04-17T06:09:26.012475+0000","last_clean":"2024-04-17T06:09:26.012475+0000","last_became_active":"2024-04-17T06:09:24.676834+0000","last_became_peered":"2024-04-17T06:09:24.676834+0000","last_unstale":"2024-04-17T06:09:26.012475+0000","last_undegraded":"2024-04-17T06:09:26.012475+0000","last_fullsized":"2024-04-17T06:09:26.012475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:25.702449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659412+0000","last_change":"2024-04-17T06:09:20.640720+0000","last_active":"2024-04-17T06:09:25.659412+0000","last_peered":"2024-04-17T06:09:25.659412+0000","last_clean":"2024-04-17T06:09:25.659412+0000","last_became_active":"2024-04-17T06:09:20.639180+0000","last_became_peered":"2024-04-17T06:09:20.639180+0000","last_unstale":"2024-04-17T06:09:25.659412+0000","last_undegraded":"2024-04-17T06:09:25.659412+0000","last_fullsized":"2024-04-17T06:09:25.659412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659510+0000","last_change":"2024-04-17T06:09:19.623937+0000","last_active":"2024-04-17T06:09:25.659510+0000","last_peered":"2024-04-17T06:09:25.659510+0000","last_clean":"2024-04-17T06:09:25.659510+0000","last_became_active":"2024-04-17T06:09:19.623613+0000","last_became_peered":"2024-04-17T06:09:19.623613+0000","last_unstale":"2024-04-17T06:09:25.659510+0000","last_undegraded":"2024-04-17T06:09:25.659510+0000","last_fullsized":"2024-04-17T06:09:25.659510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010862+0000","last_change":"2024-04-17T06:09:24.991659+0000","last_active":"2024-04-17T06:09:26.010862+0000","last_peered":"2024-04-17T06:09:26.010862+0000","last_clean":"2024-04-17T06:09:26.010862+0000","last_became_active":"2024-04-17T06:09:24.991305+0000","last_became_peered":"2024-04-17T06:09:24.991305+0000","last_unstale":"2024-04-17T06:09:26.010862+0000","last_undegraded":"2024-04-17T06:09:26.010862+0000","last_fullsized":"2024-04-17T06:09:26.010862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:22.763944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010997+0000","last_change":"2024-04-17T06:09:20.647491+0000","last_active":"2024-04-17T06:09:26.010997+0000","last_peered":"2024-04-17T06:09:26.010997+0000","last_clean":"2024-04-17T06:09:26.010997+0000","last_became_active":"2024-04-17T06:09:20.647222+0000","last_became_peered":"2024-04-17T06:09:20.647222+0000","last_unstale":"2024-04-17T06:09:26.010997+0000","last_undegraded":"2024-04-17T06:09:26.010997+0000","last_fullsized":"2024-04-17T06:09:26.010997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660425+0000","last_change":"2024-04-17T06:09:19.622734+0000","last_active":"2024-04-17T06:09:25.660425+0000","last_peered":"2024-04-17T06:09:25.660425+0000","last_clean":"2024-04-17T06:09:25.660425+0000","last_became_active":"2024-04-17T06:09:19.622401+0000","last_became_peered":"2024-04-17T06:09:19.622401+0000","last_unstale":"2024-04-17T06:09:25.660425+0000","last_undegraded":"2024-04-17T06:09:25.660425+0000","last_fullsized":"2024-04-17T06:09:25.660425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012343+0000","last_change":"2024-04-17T06:09:24.989713+0000","last_active":"2024-04-17T06:09:26.012343+0000","last_peered":"2024-04-17T06:09:26.012343+0000","last_clean":"2024-04-17T06:09:26.012343+0000","last_became_active":"2024-04-17T06:09:24.989311+0000","last_became_peered":"2024-04-17T06:09:24.989311+0000","last_unstale":"2024-04-17T06:09:26.012343+0000","last_undegraded":"2024-04-17T06:09:26.012343+0000","last_fullsized":"2024-04-17T06:09:26.012343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:09.035921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010838+0000","last_change":"2024-04-17T06:09:20.648156+0000","last_active":"2024-04-17T06:09:26.010838+0000","last_peered":"2024-04-17T06:09:26.010838+0000","last_clean":"2024-04-17T06:09:26.010838+0000","last_became_active":"2024-04-17T06:09:20.647878+0000","last_became_peered":"2024-04-17T06:09:20.647878+0000","last_unstale":"2024-04-17T06:09:26.010838+0000","last_undegraded":"2024-04-17T06:09:26.010838+0000","last_fullsized":"2024-04-17T06:09:26.010838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011779+0000","last_change":"2024-04-17T06:09:19.631943+0000","last_active":"2024-04-17T06:09:26.011779+0000","last_peered":"2024-04-17T06:09:26.011779+0000","last_clean":"2024-04-17T06:09:26.011779+0000","last_became_active":"2024-04-17T06:09:19.631287+0000","last_became_peered":"2024-04-17T06:09:19.631287+0000","last_unstale":"2024-04-17T06:09:26.011779+0000","last_undegraded":"2024-04-17T06:09:26.011779+0000","last_fullsized":"2024-04-17T06:09:26.011779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:44.893832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660591+0000","last_change":"2024-04-17T06:09:24.989665+0000","last_active":"2024-04-17T06:09:25.660591+0000","last_peered":"2024-04-17T06:09:25.660591+0000","last_clean":"2024-04-17T06:09:25.660591+0000","last_became_active":"2024-04-17T06:09:24.989241+0000","last_became_peered":"2024-04-17T06:09:24.989241+0000","last_unstale":"2024-04-17T06:09:25.660591+0000","last_undegraded":"2024-04-17T06:09:25.660591+0000","last_fullsized":"2024-04-17T06:09:25.660591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:43.835421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010532+0000","last_change":"2024-04-17T06:09:20.647147+0000","last_active":"2024-04-17T06:09:26.010532+0000","last_peered":"2024-04-17T06:09:26.010532+0000","last_clean":"2024-04-17T06:09:26.010532+0000","last_became_active":"2024-04-17T06:09:20.646657+0000","last_became_peered":"2024-04-17T06:09:20.646657+0000","last_unstale":"2024-04-17T06:09:26.010532+0000","last_undegraded":"2024-04-17T06:09:26.010532+0000","last_fullsized":"2024-04-17T06:09:26.010532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:30.596863+0000","last_change":"2024-04-17T06:09:19.629001+0000","last_active":"2024-04-17T06:09:30.596863+0000","last_peered":"2024-04-17T06:09:30.596863+0000","last_clean":"2024-04-17T06:09:30.596863+0000","last_became_active":"2024-04-17T06:09:19.628654+0000","last_became_peered":"2024-04-17T06:09:19.628654+0000","last_unstale":"2024-04-17T06:09:30.596863+0000","last_undegraded":"2024-04-17T06:09:30.596863+0000","last_fullsized":"2024-04-17T06:09:30.596863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011319+0000","last_change":"2024-04-17T06:09:24.682555+0000","last_active":"2024-04-17T06:09:26.011319+0000","last_peered":"2024-04-17T06:09:26.011319+0000","last_clean":"2024-04-17T06:09:26.011319+0000","last_became_active":"2024-04-17T06:09:24.680576+0000","last_became_peered":"2024-04-17T06:09:24.680576+0000","last_unstale":"2024-04-17T06:09:26.011319+0000","last_undegraded":"2024-04-17T06:09:26.011319+0000","last_fullsized":"2024-04-17T06:09:26.011319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:14:19.804372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010916+0000","last_change":"2024-04-17T06:09:20.644222+0000","last_active":"2024-04-17T06:09:26.010916+0000","last_peered":"2024-04-17T06:09:26.010916+0000","last_clean":"2024-04-17T06:09:26.010916+0000","last_became_active":"2024-04-17T06:09:20.643862+0000","last_became_peered":"2024-04-17T06:09:20.643862+0000","last_unstale":"2024-04-17T06:09:26.010916+0000","last_undegraded":"2024-04-17T06:09:26.010916+0000","last_fullsized":"2024-04-17T06:09:26.010916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:07.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011583+0000","last_change":"2024-04-17T06:09:19.628512+0000","last_active":"2024-04-17T06:09:26.011583+0000","last_peered":"2024-04-17T06:09:26.011583+0000","last_clean":"2024-04-17T06:09:26.011583+0000","last_became_active":"2024-04-17T06:09:19.628227+0000","last_became_peered":"2024-04-17T06:09:19.628227+0000","last_unstale":"2024-04-17T06:09:26.011583+0000","last_undegraded":"2024-04-17T06:09:26.011583+0000","last_fullsized":"2024-04-17T06:09:26.011583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011193+0000","last_change":"2024-04-17T06:09:24.682017+0000","last_active":"2024-04-17T06:09:26.011193+0000","last_peered":"2024-04-17T06:09:26.011193+0000","last_clean":"2024-04-17T06:09:26.011193+0000","last_became_active":"2024-04-17T06:09:24.681637+0000","last_became_peered":"2024-04-17T06:09:24.681637+0000","last_unstale":"2024-04-17T06:09:26.011193+0000","last_undegraded":"2024-04-17T06:09:26.011193+0000","last_fullsized":"2024-04-17T06:09:26.011193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:15.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012238+0000","last_change":"2024-04-17T06:09:20.638985+0000","last_active":"2024-04-17T06:09:26.012238+0000","last_peered":"2024-04-17T06:09:26.012238+0000","last_clean":"2024-04-17T06:09:26.012238+0000","last_became_active":"2024-04-17T06:09:20.638132+0000","last_became_peered":"2024-04-17T06:09:20.638132+0000","last_unstale":"2024-04-17T06:09:26.012238+0000","last_undegraded":"2024-04-17T06:09:26.012238+0000","last_fullsized":"2024-04-17T06:09:26.012238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660523+0000","last_change":"2024-04-17T06:09:19.621789+0000","last_active":"2024-04-17T06:09:25.660523+0000","last_peered":"2024-04-17T06:09:25.660523+0000","last_clean":"2024-04-17T06:09:25.660523+0000","last_became_active":"2024-04-17T06:09:19.621506+0000","last_became_peered":"2024-04-17T06:09:19.621506+0000","last_unstale":"2024-04-17T06:09:25.660523+0000","last_undegraded":"2024-04-17T06:09:25.660523+0000","last_fullsized":"2024-04-17T06:09:25.660523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010264+0000","last_change":"2024-04-17T06:09:24.676641+0000","last_active":"2024-04-17T06:09:26.010264+0000","last_peered":"2024-04-17T06:09:26.010264+0000","last_clean":"2024-04-17T06:09:26.010264+0000","last_became_active":"2024-04-17T06:09:24.676251+0000","last_became_peered":"2024-04-17T06:09:24.676251+0000","last_unstale":"2024-04-17T06:09:26.010264+0000","last_undegraded":"2024-04-17T06:09:26.010264+0000","last_fullsized":"2024-04-17T06:09:26.010264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:54.153325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660474+0000","last_change":"2024-04-17T06:09:20.643983+0000","last_active":"2024-04-17T06:09:25.660474+0000","last_peered":"2024-04-17T06:09:25.660474+0000","last_clean":"2024-04-17T06:09:25.660474+0000","last_became_active":"2024-04-17T06:09:20.643704+0000","last_became_peered":"2024-04-17T06:09:20.643704+0000","last_unstale":"2024-04-17T06:09:25.660474+0000","last_undegraded":"2024-04-17T06:09:25.660474+0000","last_fullsized":"2024-04-17T06:09:25.660474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660308+0000","last_change":"2024-04-17T06:09:19.626344+0000","last_active":"2024-04-17T06:09:25.660308+0000","last_peered":"2024-04-17T06:09:25.660308+0000","last_clean":"2024-04-17T06:09:25.660308+0000","last_became_active":"2024-04-17T06:09:19.626060+0000","last_became_peered":"2024-04-17T06:09:19.626060+0000","last_unstale":"2024-04-17T06:09:25.660308+0000","last_undegraded":"2024-04-17T06:09:25.660308+0000","last_fullsized":"2024-04-17T06:09:25.660308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011948+0000","last_change":"2024-04-17T06:09:24.990709+0000","last_active":"2024-04-17T06:09:26.011948+0000","last_peered":"2024-04-17T06:09:26.011948+0000","last_clean":"2024-04-17T06:09:26.011948+0000","last_became_active":"2024-04-17T06:09:24.990327+0000","last_became_peered":"2024-04-17T06:09:24.990327+0000","last_unstale":"2024-04-17T06:09:26.011948+0000","last_undegraded":"2024-04-17T06:09:26.011948+0000","last_fullsized":"2024-04-17T06:09:26.011948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:26:33.123735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012474+0000","last_change":"2024-04-17T06:09:24.994579+0000","last_active":"2024-04-17T06:09:26.012474+0000","last_peered":"2024-04-17T06:09:26.012474+0000","last_clean":"2024-04-17T06:09:26.012474+0000","last_became_active":"2024-04-17T06:09:24.994172+0000","last_became_peered":"2024-04-17T06:09:24.994172+0000","last_unstale":"2024-04-17T06:09:26.012474+0000","last_undegraded":"2024-04-17T06:09:26.012474+0000","last_fullsized":"2024-04-17T06:09:26.012474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:20.183387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010445+0000","last_change":"2024-04-17T06:09:20.646244+0000","last_active":"2024-04-17T06:09:26.010445+0000","last_peered":"2024-04-17T06:09:26.010445+0000","last_clean":"2024-04-17T06:09:26.010445+0000","last_became_active":"2024-04-17T06:09:20.645899+0000","last_became_peered":"2024-04-17T06:09:20.645899+0000","last_unstale":"2024-04-17T06:09:26.010445+0000","last_undegraded":"2024-04-17T06:09:26.010445+0000","last_fullsized":"2024-04-17T06:09:26.010445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659810+0000","last_change":"2024-04-17T06:09:19.621004+0000","last_active":"2024-04-17T06:09:25.659810+0000","last_peered":"2024-04-17T06:09:25.659810+0000","last_clean":"2024-04-17T06:09:25.659810+0000","last_became_active":"2024-04-17T06:09:19.620503+0000","last_became_peered":"2024-04-17T06:09:19.620503+0000","last_unstale":"2024-04-17T06:09:25.659810+0000","last_undegraded":"2024-04-17T06:09:25.659810+0000","last_fullsized":"2024-04-17T06:09:25.659810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660166+0000","last_change":"2024-04-17T06:09:24.674536+0000","last_active":"2024-04-17T06:09:25.660166+0000","last_peered":"2024-04-17T06:09:25.660166+0000","last_clean":"2024-04-17T06:09:25.660166+0000","last_became_active":"2024-04-17T06:09:24.674159+0000","last_became_peered":"2024-04-17T06:09:24.674159+0000","last_unstale":"2024-04-17T06:09:25.660166+0000","last_undegraded":"2024-04-17T06:09:25.660166+0000","last_fullsized":"2024-04-17T06:09:25.660166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:40.236016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010314+0000","last_change":"2024-04-17T06:09:20.638332+0000","last_active":"2024-04-17T06:09:26.010314+0000","last_peered":"2024-04-17T06:09:26.010314+0000","last_clean":"2024-04-17T06:09:26.010314+0000","last_became_active":"2024-04-17T06:09:20.638096+0000","last_became_peered":"2024-04-17T06:09:20.638096+0000","last_unstale":"2024-04-17T06:09:26.010314+0000","last_undegraded":"2024-04-17T06:09:26.010314+0000","last_fullsized":"2024-04-17T06:09:26.010314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012032+0000","last_change":"2024-04-17T06:09:19.627827+0000","last_active":"2024-04-17T06:09:26.012032+0000","last_peered":"2024-04-17T06:09:26.012032+0000","last_clean":"2024-04-17T06:09:26.012032+0000","last_became_active":"2024-04-17T06:09:19.627529+0000","last_became_peered":"2024-04-17T06:09:19.627529+0000","last_unstale":"2024-04-17T06:09:26.012032+0000","last_undegraded":"2024-04-17T06:09:26.012032+0000","last_fullsized":"2024-04-17T06:09:26.012032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659711+0000","last_change":"2024-04-17T06:09:19.623047+0000","last_active":"2024-04-17T06:09:25.659711+0000","last_peered":"2024-04-17T06:09:25.659711+0000","last_clean":"2024-04-17T06:09:25.659711+0000","last_became_active":"2024-04-17T06:09:19.622700+0000","last_became_peered":"2024-04-17T06:09:19.622700+0000","last_unstale":"2024-04-17T06:09:25.659711+0000","last_undegraded":"2024-04-17T06:09:25.659711+0000","last_fullsized":"2024-04-17T06:09:25.659711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011732+0000","last_change":"2024-04-17T06:09:20.630941+0000","last_active":"2024-04-17T06:09:26.011732+0000","last_peered":"2024-04-17T06:09:26.011732+0000","last_clean":"2024-04-17T06:09:26.011732+0000","last_became_active":"2024-04-17T06:09:20.630680+0000","last_became_peered":"2024-04-17T06:09:20.630680+0000","last_unstale":"2024-04-17T06:09:26.011732+0000","last_undegraded":"2024-04-17T06:09:26.011732+0000","last_fullsized":"2024-04-17T06:09:26.011732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661370+0000","last_change":"2024-04-17T06:09:24.668919+0000","last_active":"2024-04-17T06:09:25.661370+0000","last_peered":"2024-04-17T06:09:25.661370+0000","last_clean":"2024-04-17T06:09:25.661370+0000","last_became_active":"2024-04-17T06:09:24.668456+0000","last_became_peered":"2024-04-17T06:09:24.668456+0000","last_unstale":"2024-04-17T06:09:25.661370+0000","last_undegraded":"2024-04-17T06:09:25.661370+0000","last_fullsized":"2024-04-17T06:09:25.661370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:57.434392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011442+0000","last_change":"2024-04-17T06:09:19.622862+0000","last_active":"2024-04-17T06:09:26.011442+0000","last_peered":"2024-04-17T06:09:26.011442+0000","last_clean":"2024-04-17T06:09:26.011442+0000","last_became_active":"2024-04-17T06:09:19.622514+0000","last_became_peered":"2024-04-17T06:09:19.622514+0000","last_unstale":"2024-04-17T06:09:26.011442+0000","last_undegraded":"2024-04-17T06:09:26.011442+0000","last_fullsized":"2024-04-17T06:09:26.011442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011374+0000","last_change":"2024-04-17T06:09:20.638095+0000","last_active":"2024-04-17T06:09:26.011374+0000","last_peered":"2024-04-17T06:09:26.011374+0000","last_clean":"2024-04-17T06:09:26.011374+0000","last_became_active":"2024-04-17T06:09:20.637805+0000","last_became_peered":"2024-04-17T06:09:20.637805+0000","last_unstale":"2024-04-17T06:09:26.011374+0000","last_undegraded":"2024-04-17T06:09:26.011374+0000","last_fullsized":"2024-04-17T06:09:26.011374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:48.980595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012082+0000","last_change":"2024-04-17T06:09:24.675683+0000","last_active":"2024-04-17T06:09:26.012082+0000","last_peered":"2024-04-17T06:09:26.012082+0000","last_clean":"2024-04-17T06:09:26.012082+0000","last_became_active":"2024-04-17T06:09:24.675309+0000","last_became_peered":"2024-04-17T06:09:24.675309+0000","last_unstale":"2024-04-17T06:09:26.012082+0000","last_undegraded":"2024-04-17T06:09:26.012082+0000","last_fullsized":"2024-04-17T06:09:26.012082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:10.888244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659223+0000","last_change":"2024-04-17T06:09:20.640987+0000","last_active":"2024-04-17T06:09:25.659223+0000","last_peered":"2024-04-17T06:09:25.659223+0000","last_clean":"2024-04-17T06:09:25.659223+0000","last_became_active":"2024-04-17T06:09:20.640396+0000","last_became_peered":"2024-04-17T06:09:20.640396+0000","last_unstale":"2024-04-17T06:09:25.659223+0000","last_undegraded":"2024-04-17T06:09:25.659223+0000","last_fullsized":"2024-04-17T06:09:25.659223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660950+0000","last_change":"2024-04-17T06:09:20.642450+0000","last_active":"2024-04-17T06:09:25.660950+0000","last_peered":"2024-04-17T06:09:25.660950+0000","last_clean":"2024-04-17T06:09:25.660950+0000","last_became_active":"2024-04-17T06:09:20.639465+0000","last_became_peered":"2024-04-17T06:09:20.639465+0000","last_unstale":"2024-04-17T06:09:25.660950+0000","last_undegraded":"2024-04-17T06:09:25.660950+0000","last_fullsized":"2024-04-17T06:09:25.660950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:08.393904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660526+0000","last_change":"2024-04-17T06:09:20.652419+0000","last_active":"2024-04-17T06:09:25.660526+0000","last_peered":"2024-04-17T06:09:25.660526+0000","last_clean":"2024-04-17T06:09:25.660526+0000","last_became_active":"2024-04-17T06:09:20.652139+0000","last_became_peered":"2024-04-17T06:09:20.652139+0000","last_unstale":"2024-04-17T06:09:25.660526+0000","last_undegraded":"2024-04-17T06:09:25.660526+0000","last_fullsized":"2024-04-17T06:09:25.660526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011662+0000","last_change":"2024-04-17T06:09:20.636986+0000","last_active":"2024-04-17T06:09:26.011662+0000","last_peered":"2024-04-17T06:09:26.011662+0000","last_clean":"2024-04-17T06:09:26.011662+0000","last_became_active":"2024-04-17T06:09:20.636761+0000","last_became_peered":"2024-04-17T06:09:20.636761+0000","last_unstale":"2024-04-17T06:09:26.011662+0000","last_undegraded":"2024-04-17T06:09:26.011662+0000","last_fullsized":"2024-04-17T06:09:26.011662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012351+0000","last_change":"2024-04-17T06:09:20.646197+0000","last_active":"2024-04-17T06:09:26.012351+0000","last_peered":"2024-04-17T06:09:26.012351+0000","last_clean":"2024-04-17T06:09:26.012351+0000","last_became_active":"2024-04-17T06:09:20.645965+0000","last_became_peered":"2024-04-17T06:09:20.645965+0000","last_unstale":"2024-04-17T06:09:26.012351+0000","last_undegraded":"2024-04-17T06:09:26.012351+0000","last_fullsized":"2024-04-17T06:09:26.012351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011415+0000","last_change":"2024-04-17T06:09:20.644696+0000","last_active":"2024-04-17T06:09:26.011415+0000","last_peered":"2024-04-17T06:09:26.011415+0000","last_clean":"2024-04-17T06:09:26.011415+0000","last_became_active":"2024-04-17T06:09:20.644322+0000","last_became_peered":"2024-04-17T06:09:20.644322+0000","last_unstale":"2024-04-17T06:09:26.011415+0000","last_undegraded":"2024-04-17T06:09:26.011415+0000","last_fullsized":"2024-04-17T06:09:26.011415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660919+0000","last_change":"2024-04-17T06:09:20.650016+0000","last_active":"2024-04-17T06:09:25.660919+0000","last_peered":"2024-04-17T06:09:25.660919+0000","last_clean":"2024-04-17T06:09:25.660919+0000","last_became_active":"2024-04-17T06:09:20.649252+0000","last_became_peered":"2024-04-17T06:09:20.649252+0000","last_unstale":"2024-04-17T06:09:25.660919+0000","last_undegraded":"2024-04-17T06:09:25.660919+0000","last_fullsized":"2024-04-17T06:09:25.660919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010916+0000","last_change":"2024-04-17T06:09:20.646003+0000","last_active":"2024-04-17T06:09:26.010916+0000","last_peered":"2024-04-17T06:09:26.010916+0000","last_clean":"2024-04-17T06:09:26.010916+0000","last_became_active":"2024-04-17T06:09:20.645732+0000","last_became_peered":"2024-04-17T06:09:20.645732+0000","last_unstale":"2024-04-17T06:09:26.010916+0000","last_undegraded":"2024-04-17T06:09:26.010916+0000","last_fullsized":"2024-04-17T06:09:26.010916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659748+0000","last_change":"2024-04-17T06:09:20.650803+0000","last_active":"2024-04-17T06:09:25.659748+0000","last_peered":"2024-04-17T06:09:25.659748+0000","last_clean":"2024-04-17T06:09:25.659748+0000","last_became_active":"2024-04-17T06:09:20.649856+0000","last_became_peered":"2024-04-17T06:09:20.649856+0000","last_unstale":"2024-04-17T06:09:25.659748+0000","last_undegraded":"2024-04-17T06:09:25.659748+0000","last_fullsized":"2024-04-17T06:09:25.659748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011942+0000","last_change":"2024-04-17T06:09:20.643474+0000","last_active":"2024-04-17T06:09:26.011942+0000","last_peered":"2024-04-17T06:09:26.011942+0000","last_clean":"2024-04-17T06:09:26.011942+0000","last_became_active":"2024-04-17T06:09:20.643125+0000","last_became_peered":"2024-04-17T06:09:20.643125+0000","last_unstale":"2024-04-17T06:09:26.011942+0000","last_undegraded":"2024-04-17T06:09:26.011942+0000","last_fullsized":"2024-04-17T06:09:26.011942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660299+0000","last_change":"2024-04-17T06:09:20.646984+0000","last_active":"2024-04-17T06:09:25.660299+0000","last_peered":"2024-04-17T06:09:25.660299+0000","last_clean":"2024-04-17T06:09:25.660299+0000","last_became_active":"2024-04-17T06:09:20.646731+0000","last_became_peered":"2024-04-17T06:09:20.646731+0000","last_unstale":"2024-04-17T06:09:25.660299+0000","last_undegraded":"2024-04-17T06:09:25.660299+0000","last_fullsized":"2024-04-17T06:09:25.660299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:11.442995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011435+0000","last_change":"2024-04-17T06:09:20.648582+0000","last_active":"2024-04-17T06:09:26.011435+0000","last_peered":"2024-04-17T06:09:26.011435+0000","last_clean":"2024-04-17T06:09:26.011435+0000","last_became_active":"2024-04-17T06:09:20.648345+0000","last_became_peered":"2024-04-17T06:09:20.648345+0000","last_unstale":"2024-04-17T06:09:26.011435+0000","last_undegraded":"2024-04-17T06:09:26.011435+0000","last_fullsized":"2024-04-17T06:09:26.011435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660359+0000","last_change":"2024-04-17T06:09:20.647687+0000","last_active":"2024-04-17T06:09:25.660359+0000","last_peered":"2024-04-17T06:09:25.660359+0000","last_clean":"2024-04-17T06:09:25.660359+0000","last_became_active":"2024-04-17T06:09:20.646672+0000","last_became_peered":"2024-04-17T06:09:20.646672+0000","last_unstale":"2024-04-17T06:09:25.660359+0000","last_undegraded":"2024-04-17T06:09:25.660359+0000","last_fullsized":"2024-04-17T06:09:25.660359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011526+0000","last_change":"2024-04-17T06:09:20.646327+0000","last_active":"2024-04-17T06:09:26.011526+0000","last_peered":"2024-04-17T06:09:26.011526+0000","last_clean":"2024-04-17T06:09:26.011526+0000","last_became_active":"2024-04-17T06:09:20.645768+0000","last_became_peered":"2024-04-17T06:09:20.645768+0000","last_unstale":"2024-04-17T06:09:26.011526+0000","last_undegraded":"2024-04-17T06:09:26.011526+0000","last_fullsized":"2024-04-17T06:09:26.011526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:55.368933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661388+0000","last_change":"2024-04-17T06:09:20.638979+0000","last_active":"2024-04-17T06:09:25.661388+0000","last_peered":"2024-04-17T06:09:25.661388+0000","last_clean":"2024-04-17T06:09:25.661388+0000","last_became_active":"2024-04-17T06:09:20.638679+0000","last_became_peered":"2024-04-17T06:09:20.638679+0000","last_unstale":"2024-04-17T06:09:25.661388+0000","last_undegraded":"2024-04-17T06:09:25.661388+0000","last_fullsized":"2024-04-17T06:09:25.661388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:08.892063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661039+0000","last_change":"2024-04-17T06:09:20.647598+0000","last_active":"2024-04-17T06:09:25.661039+0000","last_peered":"2024-04-17T06:09:25.661039+0000","last_clean":"2024-04-17T06:09:25.661039+0000","last_became_active":"2024-04-17T06:09:20.647352+0000","last_became_peered":"2024-04-17T06:09:20.647352+0000","last_unstale":"2024-04-17T06:09:25.661039+0000","last_undegraded":"2024-04-17T06:09:25.661039+0000","last_fullsized":"2024-04-17T06:09:25.661039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:16.143995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659697+0000","last_change":"2024-04-17T06:09:20.634824+0000","last_active":"2024-04-17T06:09:25.659697+0000","last_peered":"2024-04-17T06:09:25.659697+0000","last_clean":"2024-04-17T06:09:25.659697+0000","last_became_active":"2024-04-17T06:09:20.634526+0000","last_became_peered":"2024-04-17T06:09:20.634526+0000","last_unstale":"2024-04-17T06:09:25.659697+0000","last_undegraded":"2024-04-17T06:09:25.659697+0000","last_fullsized":"2024-04-17T06:09:25.659697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010538+0000","last_change":"2024-04-17T06:09:20.643072+0000","last_active":"2024-04-17T06:09:26.010538+0000","last_peered":"2024-04-17T06:09:26.010538+0000","last_clean":"2024-04-17T06:09:26.010538+0000","last_became_active":"2024-04-17T06:09:20.642812+0000","last_became_peered":"2024-04-17T06:09:20.642812+0000","last_unstale":"2024-04-17T06:09:26.010538+0000","last_undegraded":"2024-04-17T06:09:26.010538+0000","last_fullsized":"2024-04-17T06:09:26.010538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010750+0000","last_change":"2024-04-17T06:09:20.636421+0000","last_active":"2024-04-17T06:09:26.010750+0000","last_peered":"2024-04-17T06:09:26.010750+0000","last_clean":"2024-04-17T06:09:26.010750+0000","last_became_active":"2024-04-17T06:09:20.636165+0000","last_became_peered":"2024-04-17T06:09:20.636165+0000","last_unstale":"2024-04-17T06:09:26.010750+0000","last_undegraded":"2024-04-17T06:09:26.010750+0000","last_fullsized":"2024-04-17T06:09:26.010750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:41.556474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011251+0000","last_change":"2024-04-17T06:09:20.639331+0000","last_active":"2024-04-17T06:09:26.011251+0000","last_peered":"2024-04-17T06:09:26.011251+0000","last_clean":"2024-04-17T06:09:26.011251+0000","last_became_active":"2024-04-17T06:09:20.639051+0000","last_became_peered":"2024-04-17T06:09:20.639051+0000","last_unstale":"2024-04-17T06:09:26.011251+0000","last_undegraded":"2024-04-17T06:09:26.011251+0000","last_fullsized":"2024-04-17T06:09:26.011251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010813+0000","last_change":"2024-04-17T06:09:20.639239+0000","last_active":"2024-04-17T06:09:26.010813+0000","last_peered":"2024-04-17T06:09:26.010813+0000","last_clean":"2024-04-17T06:09:26.010813+0000","last_became_active":"2024-04-17T06:09:20.638982+0000","last_became_peered":"2024-04-17T06:09:20.638982+0000","last_unstale":"2024-04-17T06:09:26.010813+0000","last_undegraded":"2024-04-17T06:09:26.010813+0000","last_fullsized":"2024-04-17T06:09:26.010813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:08.892063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659939+0000","last_change":"2024-04-17T06:09:20.639508+0000","last_active":"2024-04-17T06:09:25.659939+0000","last_peered":"2024-04-17T06:09:25.659939+0000","last_clean":"2024-04-17T06:09:25.659939+0000","last_became_active":"2024-04-17T06:09:20.639151+0000","last_became_peered":"2024-04-17T06:09:20.639151+0000","last_unstale":"2024-04-17T06:09:25.659939+0000","last_undegraded":"2024-04-17T06:09:25.659939+0000","last_fullsized":"2024-04-17T06:09:25.659939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660890+0000","last_change":"2024-04-17T06:09:20.639084+0000","last_active":"2024-04-17T06:09:25.660890+0000","last_peered":"2024-04-17T06:09:25.660890+0000","last_clean":"2024-04-17T06:09:25.660890+0000","last_became_active":"2024-04-17T06:09:20.638793+0000","last_became_peered":"2024-04-17T06:09:20.638793+0000","last_unstale":"2024-04-17T06:09:25.660890+0000","last_undegraded":"2024-04-17T06:09:25.660890+0000","last_fullsized":"2024-04-17T06:09:25.660890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011592+0000","last_change":"2024-04-17T06:09:20.645741+0000","last_active":"2024-04-17T06:09:26.011592+0000","last_peered":"2024-04-17T06:09:26.011592+0000","last_clean":"2024-04-17T06:09:26.011592+0000","last_became_active":"2024-04-17T06:09:20.645168+0000","last_became_peered":"2024-04-17T06:09:20.645168+0000","last_unstale":"2024-04-17T06:09:26.011592+0000","last_undegraded":"2024-04-17T06:09:26.011592+0000","last_fullsized":"2024-04-17T06:09:26.011592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:16.143995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010798+0000","last_change":"2024-04-17T06:09:20.656219+0000","last_active":"2024-04-17T06:09:26.010798+0000","last_peered":"2024-04-17T06:09:26.010798+0000","last_clean":"2024-04-17T06:09:26.010798+0000","last_became_active":"2024-04-17T06:09:20.655978+0000","last_became_peered":"2024-04-17T06:09:20.655978+0000","last_unstale":"2024-04-17T06:09:26.010798+0000","last_undegraded":"2024-04-17T06:09:26.010798+0000","last_fullsized":"2024-04-17T06:09:26.010798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010761+0000","last_change":"2024-04-17T06:09:20.650090+0000","last_active":"2024-04-17T06:09:26.010761+0000","last_peered":"2024-04-17T06:09:26.010761+0000","last_clean":"2024-04-17T06:09:26.010761+0000","last_became_active":"2024-04-17T06:09:20.649596+0000","last_became_peered":"2024-04-17T06:09:20.649596+0000","last_unstale":"2024-04-17T06:09:26.010761+0000","last_undegraded":"2024-04-17T06:09:26.010761+0000","last_fullsized":"2024-04-17T06:09:26.010761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660177+0000","last_change":"2024-04-17T06:09:20.634204+0000","last_active":"2024-04-17T06:09:25.660177+0000","last_peered":"2024-04-17T06:09:25.660177+0000","last_clean":"2024-04-17T06:09:25.660177+0000","last_became_active":"2024-04-17T06:09:20.633924+0000","last_became_peered":"2024-04-17T06:09:20.633924+0000","last_unstale":"2024-04-17T06:09:25.660177+0000","last_undegraded":"2024-04-17T06:09:25.660177+0000","last_fullsized":"2024-04-17T06:09:25.660177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010796+0000","last_change":"2024-04-17T06:09:20.645437+0000","last_active":"2024-04-17T06:09:26.010796+0000","last_peered":"2024-04-17T06:09:26.010796+0000","last_clean":"2024-04-17T06:09:26.010796+0000","last_became_active":"2024-04-17T06:09:20.645015+0000","last_became_peered":"2024-04-17T06:09:20.645015+0000","last_unstale":"2024-04-17T06:09:26.010796+0000","last_undegraded":"2024-04-17T06:09:26.010796+0000","last_fullsized":"2024-04-17T06:09:26.010796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660367+0000","last_change":"2024-04-17T06:09:20.650436+0000","last_active":"2024-04-17T06:09:25.660367+0000","last_peered":"2024-04-17T06:09:25.660367+0000","last_clean":"2024-04-17T06:09:25.660367+0000","last_became_active":"2024-04-17T06:09:20.650132+0000","last_became_peered":"2024-04-17T06:09:20.650132+0000","last_unstale":"2024-04-17T06:09:25.660367+0000","last_undegraded":"2024-04-17T06:09:25.660367+0000","last_fullsized":"2024-04-17T06:09:25.660367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011167+0000","last_change":"2024-04-17T06:09:20.634255+0000","last_active":"2024-04-17T06:09:26.011167+0000","last_peered":"2024-04-17T06:09:26.011167+0000","last_clean":"2024-04-17T06:09:26.011167+0000","last_became_active":"2024-04-17T06:09:20.633879+0000","last_became_peered":"2024-04-17T06:09:20.633879+0000","last_unstale":"2024-04-17T06:09:26.011167+0000","last_undegraded":"2024-04-17T06:09:26.011167+0000","last_fullsized":"2024-04-17T06:09:26.011167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:07.365672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010938+0000","last_change":"2024-04-17T06:09:20.635023+0000","last_active":"2024-04-17T06:09:26.010938+0000","last_peered":"2024-04-17T06:09:26.010938+0000","last_clean":"2024-04-17T06:09:26.010938+0000","last_became_active":"2024-04-17T06:09:20.634245+0000","last_became_peered":"2024-04-17T06:09:20.634245+0000","last_unstale":"2024-04-17T06:09:26.010938+0000","last_undegraded":"2024-04-17T06:09:26.010938+0000","last_fullsized":"2024-04-17T06:09:26.010938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011506+0000","last_change":"2024-04-17T06:09:20.653636+0000","last_active":"2024-04-17T06:09:26.011506+0000","last_peered":"2024-04-17T06:09:26.011506+0000","last_clean":"2024-04-17T06:09:26.011506+0000","last_became_active":"2024-04-17T06:09:20.653366+0000","last_became_peered":"2024-04-17T06:09:20.653366+0000","last_unstale":"2024-04-17T06:09:26.011506+0000","last_undegraded":"2024-04-17T06:09:26.011506+0000","last_fullsized":"2024-04-17T06:09:26.011506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:58.610924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T06:09:42.097 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T06:09:42.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.237+0000 7f8395088640 1 Processor -- start 2024-04-17T06:09:42.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.237+0000 7f8395088640 1 -- start start 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.237+0000 7f8395088640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f839015ec00 0x7f83900d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.237+0000 7f8395088640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d43d0 0x7f83900d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.237+0000 7f8395088640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f83900d1810 0x7f83900730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.237+0000 7f8395088640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f839015efe0 con 0x7f839015ec00 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.237+0000 7f8395088640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f83900d3f40 con 0x7f83900d1810 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.237+0000 7f8395088640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f83900d1c70 con 0x7f83900d43d0 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f839015ec00 0x7f83900d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f839015ec00 0x7f83900d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49030/0 (socket says 172.21.15.135:49030) 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838ed9d640 1 -- 172.21.15.135:0/230045932 learned_addr learned my addr 172.21.15.135:0/230045932 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838e59c640 1 --2- 172.21.15.135:0/230045932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d43d0 0x7f83900d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838f59e640 1 --2- 172.21.15.135:0/230045932 >> v2:172.21.15.190:3300/0 conn(0x7f83900d1810 0x7f83900730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838ed9d640 1 -- 172.21.15.135:0/230045932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d43d0 msgr2=0x7f83900d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838ed9d640 1 --2- 172.21.15.135:0/230045932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d43d0 0x7f83900d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838ed9d640 1 -- 172.21.15.135:0/230045932 >> v2:172.21.15.190:3300/0 conn(0x7f83900d1810 msgr2=0x7f83900730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838ed9d640 1 --2- 172.21.15.135:0/230045932 >> v2:172.21.15.190:3300/0 conn(0x7f83900d1810 0x7f83900730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.238+0000 7f838ed9d640 1 -- 172.21.15.135:0/230045932 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83900d3330 con 0x7f839015ec00 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f838ed9d640 1 --2- 172.21.15.135:0/230045932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f83900d2990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f839006dd80 tx=0x7f837802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3220a62b330548 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f838dd9b640 1 -- 172.21.15.135:0/230045932 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378034070 con 0x7f839015ec00 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f838dd9b640 1 -- 172.21.15.135:0/230045932 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8378030860 con 0x7f839015ec00 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f838dd9b640 1 -- 172.21.15.135:0/230045932 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8378030c30 con 0x7f839015ec00 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 -- 172.21.15.135:0/230045932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 msgr2=0x7f83900d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 --2- 172.21.15.135:0/230045932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f83900d2990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f839006dd80 tx=0x7f837802b9a0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 -- 172.21.15.135:0/230045932 shutdown_connections 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 --2- 172.21.15.135:0/230045932 >> v2:172.21.15.190:3300/0 conn(0x7f83900d1810 0x7f83900730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 --2- 172.21.15.135:0/230045932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d43d0 0x7f83900d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 --2- 172.21.15.135:0/230045932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f83900d2990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 -- 172.21.15.135:0/230045932 >> 172.21.15.135:0/230045932 conn(0x7f839006cc90 msgr2=0x7f839006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:42.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 -- 172.21.15.135:0/230045932 shutdown_connections 2024-04-17T06:09:42.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.239+0000 7f8395088640 1 -- 172.21.15.135:0/230045932 wait complete. 2024-04-17T06:09:42.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.241+0000 7f8395088640 1 Processor -- start 2024-04-17T06:09:42.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.241+0000 7f8395088640 1 -- start start 2024-04-17T06:09:42.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f8395088640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d1810 0x7f83900cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f8395088640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83900d43d0 0x7f839014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f8395088640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f839014ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f8395088640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83900d3bd0 con 0x7f839015ec00 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f8395088640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f83900d38f0 con 0x7f83900d43d0 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f8395088640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f83900d2ed0 con 0x7f83900d1810 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f839014ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f839014ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49036/0 (socket says 172.21.15.135:49036) 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 -- 172.21.15.135:0/1297108604 learned_addr learned my addr 172.21.15.135:0/1297108604 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838ed9d640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d1810 0x7f83900cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838e59c640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83900d43d0 0x7f839014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 -- 172.21.15.135:0/1297108604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d1810 msgr2=0x7f83900cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d1810 0x7f83900cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 -- 172.21.15.135:0/1297108604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83900d43d0 msgr2=0x7f839014e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83900d43d0 0x7f839014e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 -- 172.21.15.135:0/1297108604 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8390160000 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f838f59e640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f839014ee80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f838800eed0 tx=0x7f838800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f83877fe640 1 -- 172.21.15.135:0/1297108604 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388018020 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f83877fe640 1 -- 172.21.15.135:0/1297108604 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8388014400 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83901602e0 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f83877fe640 1 -- 172.21.15.135:0/1297108604 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83880147f0 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.242+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83901608a0 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.243+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8390074940 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.244+0000 7f83877fe640 1 -- 172.21.15.135:0/1297108604 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8388005d10 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.244+0000 7f83877fe640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f837006c8f0 0x7f837006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.244+0000 7f83877fe640 1 -- 172.21.15.135:0/1297108604 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f838808c3c0 con 0x7f839015ec00 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.244+0000 7f838ed9d640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f837006c8f0 0x7f837006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.245+0000 7f838ed9d640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f837006c8f0 0x7f837006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8378036000 tx=0x7f837803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.249+0000 7f83877fe640 1 -- 172.21.15.135:0/1297108604 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8388022070 con 0x7f839015ec00 2024-04-17T06:09:42.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.422+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 --> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83900b47d0 con 0x7f837006c8f0 2024-04-17T06:09:42.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.430+0000 7f83877fe640 1 -- 172.21.15.135:0/1297108604 <== mgr.4113 v2:172.21.15.135:6832/3190814219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f83900b47d0 con 0x7f837006c8f0 2024-04-17T06:09:42.432 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:42.434 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-17T06:09:42.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f837006c8f0 msgr2=0x7f837006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f837006c8f0 0x7f837006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8378036000 tx=0x7f837803b040 comp rx=0 tx=0).stop 2024-04-17T06:09:42.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 msgr2=0x7f839014ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f839014ee80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f838800eed0 tx=0x7f838800b480 comp rx=0 tx=0).stop 2024-04-17T06:09:42.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 shutdown_connections 2024-04-17T06:09:42.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f837006c8f0 0x7f837006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f839015ec00 0x7f839014ee80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f83900d43d0 0x7f839014e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 --2- 172.21.15.135:0/1297108604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f83900d1810 0x7f83900cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 >> 172.21.15.135:0/1297108604 conn(0x7f839006cc90 msgr2=0x7f83900cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:42.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 shutdown_connections 2024-04-17T06:09:42.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.434+0000 7f8395088640 1 -- 172.21.15.135:0/1297108604 wait complete. 2024-04-17T06:09:42.443 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-17T06:09:41.942268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219384,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755336,"statfs":{"total":773094113280,"available":772869464064,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002585"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012356+0000","last_change":"2024-04-17T06:09:19.632439+0000","last_active":"2024-04-17T06:09:26.012356+0000","last_peered":"2024-04-17T06:09:26.012356+0000","last_clean":"2024-04-17T06:09:26.012356+0000","last_became_active":"2024-04-17T06:09:19.632170+0000","last_became_peered":"2024-04-17T06:09:19.632170+0000","last_unstale":"2024-04-17T06:09:26.012356+0000","last_undegraded":"2024-04-17T06:09:26.012356+0000","last_fullsized":"2024-04-17T06:09:26.012356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:58.547080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011716+0000","last_change":"2024-04-17T06:09:20.647290+0000","last_active":"2024-04-17T06:09:26.011716+0000","last_peered":"2024-04-17T06:09:26.011716+0000","last_clean":"2024-04-17T06:09:26.011716+0000","last_became_active":"2024-04-17T06:09:20.646950+0000","last_became_peered":"2024-04-17T06:09:20.646950+0000","last_unstale":"2024-04-17T06:09:26.011716+0000","last_undegraded":"2024-04-17T06:09:26.011716+0000","last_fullsized":"2024-04-17T06:09:26.011716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011791+0000","last_change":"2024-04-17T06:09:24.992961+0000","last_active":"2024-04-17T06:09:26.011791+0000","last_peered":"2024-04-17T06:09:26.011791+0000","last_clean":"2024-04-17T06:09:26.011791+0000","last_became_active":"2024-04-17T06:09:24.992540+0000","last_became_peered":"2024-04-17T06:09:24.992540+0000","last_unstale":"2024-04-17T06:09:26.011791+0000","last_undegraded":"2024-04-17T06:09:26.011791+0000","last_fullsized":"2024-04-17T06:09:26.011791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:03:42.943499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011930+0000","last_change":"2024-04-17T06:09:19.632406+0000","last_active":"2024-04-17T06:09:26.011930+0000","last_peered":"2024-04-17T06:09:26.011930+0000","last_clean":"2024-04-17T06:09:26.011930+0000","last_became_active":"2024-04-17T06:09:19.631990+0000","last_became_peered":"2024-04-17T06:09:19.631990+0000","last_unstale":"2024-04-17T06:09:26.011930+0000","last_undegraded":"2024-04-17T06:09:26.011930+0000","last_fullsized":"2024-04-17T06:09:26.011930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:35.753743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012065+0000","last_change":"2024-04-17T06:09:20.631799+0000","last_active":"2024-04-17T06:09:26.012065+0000","last_peered":"2024-04-17T06:09:26.012065+0000","last_clean":"2024-04-17T06:09:26.012065+0000","last_became_active":"2024-04-17T06:09:20.631551+0000","last_became_peered":"2024-04-17T06:09:20.631551+0000","last_unstale":"2024-04-17T06:09:26.012065+0000","last_undegraded":"2024-04-17T06:09:26.012065+0000","last_fullsized":"2024-04-17T06:09:26.012065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011413+0000","last_change":"2024-04-17T06:09:24.681573+0000","last_active":"2024-04-17T06:09:26.011413+0000","last_peered":"2024-04-17T06:09:26.011413+0000","last_clean":"2024-04-17T06:09:26.011413+0000","last_became_active":"2024-04-17T06:09:24.681194+0000","last_became_peered":"2024-04-17T06:09:24.681194+0000","last_unstale":"2024-04-17T06:09:26.011413+0000","last_undegraded":"2024-04-17T06:09:26.011413+0000","last_fullsized":"2024-04-17T06:09:26.011413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010848+0000","last_change":"2024-04-17T06:09:19.626563+0000","last_active":"2024-04-17T06:09:26.010848+0000","last_peered":"2024-04-17T06:09:26.010848+0000","last_clean":"2024-04-17T06:09:26.010848+0000","last_became_active":"2024-04-17T06:09:19.626169+0000","last_became_peered":"2024-04-17T06:09:19.626169+0000","last_unstale":"2024-04-17T06:09:26.010848+0000","last_undegraded":"2024-04-17T06:09:26.010848+0000","last_fullsized":"2024-04-17T06:09:26.010848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:07.389883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660478+0000","last_change":"2024-04-17T06:09:20.629675+0000","last_active":"2024-04-17T06:09:25.660478+0000","last_peered":"2024-04-17T06:09:25.660478+0000","last_clean":"2024-04-17T06:09:25.660478+0000","last_became_active":"2024-04-17T06:09:20.629335+0000","last_became_peered":"2024-04-17T06:09:20.629335+0000","last_unstale":"2024-04-17T06:09:25.660478+0000","last_undegraded":"2024-04-17T06:09:25.660478+0000","last_fullsized":"2024-04-17T06:09:25.660478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010974+0000","last_change":"2024-04-17T06:09:24.681484+0000","last_active":"2024-04-17T06:09:26.010974+0000","last_peered":"2024-04-17T06:09:26.010974+0000","last_clean":"2024-04-17T06:09:26.010974+0000","last_became_active":"2024-04-17T06:09:24.681074+0000","last_became_peered":"2024-04-17T06:09:24.681074+0000","last_unstale":"2024-04-17T06:09:26.010974+0000","last_undegraded":"2024-04-17T06:09:26.010974+0000","last_fullsized":"2024-04-17T06:09:26.010974+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:21.508074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011016+0000","last_change":"2024-04-17T06:09:19.629666+0000","last_active":"2024-04-17T06:09:26.011016+0000","last_peered":"2024-04-17T06:09:26.011016+0000","last_clean":"2024-04-17T06:09:26.011016+0000","last_became_active":"2024-04-17T06:09:19.629414+0000","last_became_peered":"2024-04-17T06:09:19.629414+0000","last_unstale":"2024-04-17T06:09:26.011016+0000","last_undegraded":"2024-04-17T06:09:26.011016+0000","last_fullsized":"2024-04-17T06:09:26.011016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660727+0000","last_change":"2024-04-17T06:09:20.629643+0000","last_active":"2024-04-17T06:09:25.660727+0000","last_peered":"2024-04-17T06:09:25.660727+0000","last_clean":"2024-04-17T06:09:25.660727+0000","last_became_active":"2024-04-17T06:09:20.629349+0000","last_became_peered":"2024-04-17T06:09:20.629349+0000","last_unstale":"2024-04-17T06:09:25.660727+0000","last_undegraded":"2024-04-17T06:09:25.660727+0000","last_fullsized":"2024-04-17T06:09:25.660727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:55.368933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011234+0000","last_change":"2024-04-17T06:09:24.673766+0000","last_active":"2024-04-17T06:09:26.011234+0000","last_peered":"2024-04-17T06:09:26.011234+0000","last_clean":"2024-04-17T06:09:26.011234+0000","last_became_active":"2024-04-17T06:09:24.673384+0000","last_became_peered":"2024-04-17T06:09:24.673384+0000","last_unstale":"2024-04-17T06:09:26.011234+0000","last_undegraded":"2024-04-17T06:09:26.011234+0000","last_fullsized":"2024-04-17T06:09:26.011234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:40.236016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659389+0000","last_change":"2024-04-17T06:09:20.643805+0000","last_active":"2024-04-17T06:09:25.659389+0000","last_peered":"2024-04-17T06:09:25.659389+0000","last_clean":"2024-04-17T06:09:25.659389+0000","last_became_active":"2024-04-17T06:09:20.643199+0000","last_became_peered":"2024-04-17T06:09:20.643199+0000","last_unstale":"2024-04-17T06:09:25.659389+0000","last_undegraded":"2024-04-17T06:09:25.659389+0000","last_fullsized":"2024-04-17T06:09:25.659389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:07.365672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011214+0000","last_change":"2024-04-17T06:09:19.626503+0000","last_active":"2024-04-17T06:09:26.011214+0000","last_peered":"2024-04-17T06:09:26.011214+0000","last_clean":"2024-04-17T06:09:26.011214+0000","last_became_active":"2024-04-17T06:09:19.625556+0000","last_became_peered":"2024-04-17T06:09:19.625556+0000","last_unstale":"2024-04-17T06:09:26.011214+0000","last_undegraded":"2024-04-17T06:09:26.011214+0000","last_fullsized":"2024-04-17T06:09:26.011214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010558+0000","last_change":"2024-04-17T06:09:24.680158+0000","last_active":"2024-04-17T06:09:26.010558+0000","last_peered":"2024-04-17T06:09:26.010558+0000","last_clean":"2024-04-17T06:09:26.010558+0000","last_became_active":"2024-04-17T06:09:24.679694+0000","last_became_peered":"2024-04-17T06:09:24.679694+0000","last_unstale":"2024-04-17T06:09:26.010558+0000","last_undegraded":"2024-04-17T06:09:26.010558+0000","last_fullsized":"2024-04-17T06:09:26.010558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:38.282988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011354+0000","last_change":"2024-04-17T06:09:20.638610+0000","last_active":"2024-04-17T06:09:26.011354+0000","last_peered":"2024-04-17T06:09:26.011354+0000","last_clean":"2024-04-17T06:09:26.011354+0000","last_became_active":"2024-04-17T06:09:20.638275+0000","last_became_peered":"2024-04-17T06:09:20.638275+0000","last_unstale":"2024-04-17T06:09:26.011354+0000","last_undegraded":"2024-04-17T06:09:26.011354+0000","last_fullsized":"2024-04-17T06:09:26.011354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011423+0000","last_change":"2024-04-17T06:09:19.626821+0000","last_active":"2024-04-17T06:09:26.011423+0000","last_peered":"2024-04-17T06:09:26.011423+0000","last_clean":"2024-04-17T06:09:26.011423+0000","last_became_active":"2024-04-17T06:09:19.626550+0000","last_became_peered":"2024-04-17T06:09:19.626550+0000","last_unstale":"2024-04-17T06:09:26.011423+0000","last_undegraded":"2024-04-17T06:09:26.011423+0000","last_fullsized":"2024-04-17T06:09:26.011423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010388+0000","last_change":"2024-04-17T06:09:24.680040+0000","last_active":"2024-04-17T06:09:26.010388+0000","last_peered":"2024-04-17T06:09:26.010388+0000","last_clean":"2024-04-17T06:09:26.010388+0000","last_became_active":"2024-04-17T06:09:24.679567+0000","last_became_peered":"2024-04-17T06:09:24.679567+0000","last_unstale":"2024-04-17T06:09:26.010388+0000","last_undegraded":"2024-04-17T06:09:26.010388+0000","last_fullsized":"2024-04-17T06:09:26.010388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:21.549944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010400+0000","last_change":"2024-04-17T06:09:20.645617+0000","last_active":"2024-04-17T06:09:26.010400+0000","last_peered":"2024-04-17T06:09:26.010400+0000","last_clean":"2024-04-17T06:09:26.010400+0000","last_became_active":"2024-04-17T06:09:20.645205+0000","last_became_peered":"2024-04-17T06:09:20.645205+0000","last_unstale":"2024-04-17T06:09:26.010400+0000","last_undegraded":"2024-04-17T06:09:26.010400+0000","last_fullsized":"2024-04-17T06:09:26.010400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010623+0000","last_change":"2024-04-17T06:09:19.623695+0000","last_active":"2024-04-17T06:09:26.010623+0000","last_peered":"2024-04-17T06:09:26.010623+0000","last_clean":"2024-04-17T06:09:26.010623+0000","last_became_active":"2024-04-17T06:09:19.623442+0000","last_became_peered":"2024-04-17T06:09:19.623442+0000","last_unstale":"2024-04-17T06:09:26.010623+0000","last_undegraded":"2024-04-17T06:09:26.010623+0000","last_fullsized":"2024-04-17T06:09:26.010623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:07.888042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010887+0000","last_change":"2024-04-17T06:09:24.990476+0000","last_active":"2024-04-17T06:09:26.010887+0000","last_peered":"2024-04-17T06:09:26.010887+0000","last_clean":"2024-04-17T06:09:26.010887+0000","last_became_active":"2024-04-17T06:09:24.990092+0000","last_became_peered":"2024-04-17T06:09:24.990092+0000","last_unstale":"2024-04-17T06:09:26.010887+0000","last_undegraded":"2024-04-17T06:09:26.010887+0000","last_fullsized":"2024-04-17T06:09:26.010887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010691+0000","last_change":"2024-04-17T06:09:20.633239+0000","last_active":"2024-04-17T06:09:26.010691+0000","last_peered":"2024-04-17T06:09:26.010691+0000","last_clean":"2024-04-17T06:09:26.010691+0000","last_became_active":"2024-04-17T06:09:20.632946+0000","last_became_peered":"2024-04-17T06:09:20.632946+0000","last_unstale":"2024-04-17T06:09:26.010691+0000","last_undegraded":"2024-04-17T06:09:26.010691+0000","last_fullsized":"2024-04-17T06:09:26.010691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:01.439932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010236+0000","last_change":"2024-04-17T06:09:19.632434+0000","last_active":"2024-04-17T06:09:26.010236+0000","last_peered":"2024-04-17T06:09:26.010236+0000","last_clean":"2024-04-17T06:09:26.010236+0000","last_became_active":"2024-04-17T06:09:19.632174+0000","last_became_peered":"2024-04-17T06:09:19.632174+0000","last_unstale":"2024-04-17T06:09:26.010236+0000","last_undegraded":"2024-04-17T06:09:26.010236+0000","last_fullsized":"2024-04-17T06:09:26.010236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:06.724008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010716+0000","last_change":"2024-04-17T06:09:24.680395+0000","last_active":"2024-04-17T06:09:26.010716+0000","last_peered":"2024-04-17T06:09:26.010716+0000","last_clean":"2024-04-17T06:09:26.010716+0000","last_became_active":"2024-04-17T06:09:24.679895+0000","last_became_peered":"2024-04-17T06:09:24.679895+0000","last_unstale":"2024-04-17T06:09:26.010716+0000","last_undegraded":"2024-04-17T06:09:26.010716+0000","last_fullsized":"2024-04-17T06:09:26.010716+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:14.465883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012593+0000","last_change":"2024-04-17T06:09:20.638642+0000","last_active":"2024-04-17T06:09:26.012593+0000","last_peered":"2024-04-17T06:09:26.012593+0000","last_clean":"2024-04-17T06:09:26.012593+0000","last_became_active":"2024-04-17T06:09:20.638351+0000","last_became_peered":"2024-04-17T06:09:20.638351+0000","last_unstale":"2024-04-17T06:09:26.012593+0000","last_undegraded":"2024-04-17T06:09:26.012593+0000","last_fullsized":"2024-04-17T06:09:26.012593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660496+0000","last_change":"2024-04-17T06:09:19.622032+0000","last_active":"2024-04-17T06:09:25.660496+0000","last_peered":"2024-04-17T06:09:25.660496+0000","last_clean":"2024-04-17T06:09:25.660496+0000","last_became_active":"2024-04-17T06:09:19.621765+0000","last_became_peered":"2024-04-17T06:09:19.621765+0000","last_unstale":"2024-04-17T06:09:25.660496+0000","last_undegraded":"2024-04-17T06:09:25.660496+0000","last_fullsized":"2024-04-17T06:09:25.660496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010414+0000","last_change":"2024-04-17T06:09:24.990792+0000","last_active":"2024-04-17T06:09:26.010414+0000","last_peered":"2024-04-17T06:09:26.010414+0000","last_clean":"2024-04-17T06:09:26.010414+0000","last_became_active":"2024-04-17T06:09:24.990371+0000","last_became_peered":"2024-04-17T06:09:24.990371+0000","last_unstale":"2024-04-17T06:09:26.010414+0000","last_undegraded":"2024-04-17T06:09:26.010414+0000","last_fullsized":"2024-04-17T06:09:26.010414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:26.189133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011005+0000","last_change":"2024-04-17T06:09:20.648546+0000","last_active":"2024-04-17T06:09:26.011005+0000","last_peered":"2024-04-17T06:09:26.011005+0000","last_clean":"2024-04-17T06:09:26.011005+0000","last_became_active":"2024-04-17T06:09:20.647853+0000","last_became_peered":"2024-04-17T06:09:20.647853+0000","last_unstale":"2024-04-17T06:09:26.011005+0000","last_undegraded":"2024-04-17T06:09:26.011005+0000","last_fullsized":"2024-04-17T06:09:26.011005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011025+0000","last_change":"2024-04-17T06:09:19.627150+0000","last_active":"2024-04-17T06:09:26.011025+0000","last_peered":"2024-04-17T06:09:26.011025+0000","last_clean":"2024-04-17T06:09:26.011025+0000","last_became_active":"2024-04-17T06:09:19.626875+0000","last_became_peered":"2024-04-17T06:09:19.626875+0000","last_unstale":"2024-04-17T06:09:26.011025+0000","last_undegraded":"2024-04-17T06:09:26.011025+0000","last_fullsized":"2024-04-17T06:09:26.011025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010576+0000","last_change":"2024-04-17T06:09:24.992298+0000","last_active":"2024-04-17T06:09:26.010576+0000","last_peered":"2024-04-17T06:09:26.010576+0000","last_clean":"2024-04-17T06:09:26.010576+0000","last_became_active":"2024-04-17T06:09:24.991871+0000","last_became_peered":"2024-04-17T06:09:24.991871+0000","last_unstale":"2024-04-17T06:09:26.010576+0000","last_undegraded":"2024-04-17T06:09:26.010576+0000","last_fullsized":"2024-04-17T06:09:26.010576+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:43.835421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011016+0000","last_change":"2024-04-17T06:09:20.639206+0000","last_active":"2024-04-17T06:09:26.011016+0000","last_peered":"2024-04-17T06:09:26.011016+0000","last_clean":"2024-04-17T06:09:26.011016+0000","last_became_active":"2024-04-17T06:09:20.638954+0000","last_became_peered":"2024-04-17T06:09:20.638954+0000","last_unstale":"2024-04-17T06:09:26.011016+0000","last_undegraded":"2024-04-17T06:09:26.011016+0000","last_fullsized":"2024-04-17T06:09:26.011016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011535+0000","last_change":"2024-04-17T06:09:19.632347+0000","last_active":"2024-04-17T06:09:26.011535+0000","last_peered":"2024-04-17T06:09:26.011535+0000","last_clean":"2024-04-17T06:09:26.011535+0000","last_became_active":"2024-04-17T06:09:19.632117+0000","last_became_peered":"2024-04-17T06:09:19.632117+0000","last_unstale":"2024-04-17T06:09:26.011535+0000","last_undegraded":"2024-04-17T06:09:26.011535+0000","last_fullsized":"2024-04-17T06:09:26.011535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:07.888042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011384+0000","last_change":"2024-04-17T06:09:24.989732+0000","last_active":"2024-04-17T06:09:26.011384+0000","last_peered":"2024-04-17T06:09:26.011384+0000","last_clean":"2024-04-17T06:09:26.011384+0000","last_became_active":"2024-04-17T06:09:24.989320+0000","last_became_peered":"2024-04-17T06:09:24.989320+0000","last_unstale":"2024-04-17T06:09:26.011384+0000","last_undegraded":"2024-04-17T06:09:26.011384+0000","last_fullsized":"2024-04-17T06:09:26.011384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:59.408326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660737+0000","last_change":"2024-04-17T06:09:20.652167+0000","last_active":"2024-04-17T06:09:25.660737+0000","last_peered":"2024-04-17T06:09:25.660737+0000","last_clean":"2024-04-17T06:09:25.660737+0000","last_became_active":"2024-04-17T06:09:20.651908+0000","last_became_peered":"2024-04-17T06:09:20.651908+0000","last_unstale":"2024-04-17T06:09:25.660737+0000","last_undegraded":"2024-04-17T06:09:25.660737+0000","last_fullsized":"2024-04-17T06:09:25.660737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:58.610924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011096+0000","last_change":"2024-04-17T06:09:19.631750+0000","last_active":"2024-04-17T06:09:26.011096+0000","last_peered":"2024-04-17T06:09:26.011096+0000","last_clean":"2024-04-17T06:09:26.011096+0000","last_became_active":"2024-04-17T06:09:19.631463+0000","last_became_peered":"2024-04-17T06:09:19.631463+0000","last_unstale":"2024-04-17T06:09:26.011096+0000","last_undegraded":"2024-04-17T06:09:26.011096+0000","last_fullsized":"2024-04-17T06:09:26.011096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:07.389883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011105+0000","last_change":"2024-04-17T06:09:24.680787+0000","last_active":"2024-04-17T06:09:26.011105+0000","last_peered":"2024-04-17T06:09:26.011105+0000","last_clean":"2024-04-17T06:09:26.011105+0000","last_became_active":"2024-04-17T06:09:24.680279+0000","last_became_peered":"2024-04-17T06:09:24.680279+0000","last_unstale":"2024-04-17T06:09:26.011105+0000","last_undegraded":"2024-04-17T06:09:26.011105+0000","last_fullsized":"2024-04-17T06:09:26.011105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:25.702449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011262+0000","last_change":"2024-04-17T06:09:20.632789+0000","last_active":"2024-04-17T06:09:26.011262+0000","last_peered":"2024-04-17T06:09:26.011262+0000","last_clean":"2024-04-17T06:09:26.011262+0000","last_became_active":"2024-04-17T06:09:20.632199+0000","last_became_peered":"2024-04-17T06:09:20.632199+0000","last_unstale":"2024-04-17T06:09:26.011262+0000","last_undegraded":"2024-04-17T06:09:26.011262+0000","last_fullsized":"2024-04-17T06:09:26.011262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:08.393904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660530+0000","last_change":"2024-04-17T06:09:19.624615+0000","last_active":"2024-04-17T06:09:25.660530+0000","last_peered":"2024-04-17T06:09:25.660530+0000","last_clean":"2024-04-17T06:09:25.660530+0000","last_became_active":"2024-04-17T06:09:19.624333+0000","last_became_peered":"2024-04-17T06:09:19.624333+0000","last_unstale":"2024-04-17T06:09:25.660530+0000","last_undegraded":"2024-04-17T06:09:25.660530+0000","last_fullsized":"2024-04-17T06:09:25.660530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011059+0000","last_change":"2024-04-17T06:09:24.675868+0000","last_active":"2024-04-17T06:09:26.011059+0000","last_peered":"2024-04-17T06:09:26.011059+0000","last_clean":"2024-04-17T06:09:26.011059+0000","last_became_active":"2024-04-17T06:09:24.675269+0000","last_became_peered":"2024-04-17T06:09:24.675269+0000","last_unstale":"2024-04-17T06:09:26.011059+0000","last_undegraded":"2024-04-17T06:09:26.011059+0000","last_fullsized":"2024-04-17T06:09:26.011059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:02.650317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010687+0000","last_change":"2024-04-17T06:09:20.638626+0000","last_active":"2024-04-17T06:09:26.010687+0000","last_peered":"2024-04-17T06:09:26.010687+0000","last_clean":"2024-04-17T06:09:26.010687+0000","last_became_active":"2024-04-17T06:09:20.638339+0000","last_became_peered":"2024-04-17T06:09:20.638339+0000","last_unstale":"2024-04-17T06:09:26.010687+0000","last_undegraded":"2024-04-17T06:09:26.010687+0000","last_fullsized":"2024-04-17T06:09:26.010687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661071+0000","last_change":"2024-04-17T06:09:19.621292+0000","last_active":"2024-04-17T06:09:25.661071+0000","last_peered":"2024-04-17T06:09:25.661071+0000","last_clean":"2024-04-17T06:09:25.661071+0000","last_became_active":"2024-04-17T06:09:19.620827+0000","last_became_peered":"2024-04-17T06:09:19.620827+0000","last_unstale":"2024-04-17T06:09:25.661071+0000","last_undegraded":"2024-04-17T06:09:25.661071+0000","last_fullsized":"2024-04-17T06:09:25.661071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011064+0000","last_change":"2024-04-17T06:09:24.682574+0000","last_active":"2024-04-17T06:09:26.011064+0000","last_peered":"2024-04-17T06:09:26.011064+0000","last_clean":"2024-04-17T06:09:26.011064+0000","last_became_active":"2024-04-17T06:09:24.680550+0000","last_became_peered":"2024-04-17T06:09:24.680550+0000","last_unstale":"2024-04-17T06:09:26.011064+0000","last_undegraded":"2024-04-17T06:09:26.011064+0000","last_fullsized":"2024-04-17T06:09:26.011064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:20.183387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010301+0000","last_change":"2024-04-17T06:09:20.630308+0000","last_active":"2024-04-17T06:09:26.010301+0000","last_peered":"2024-04-17T06:09:26.010301+0000","last_clean":"2024-04-17T06:09:26.010301+0000","last_became_active":"2024-04-17T06:09:20.630052+0000","last_became_peered":"2024-04-17T06:09:20.630052+0000","last_unstale":"2024-04-17T06:09:26.010301+0000","last_undegraded":"2024-04-17T06:09:26.010301+0000","last_fullsized":"2024-04-17T06:09:26.010301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:53.394999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010349+0000","last_change":"2024-04-17T06:09:19.624643+0000","last_active":"2024-04-17T06:09:26.010349+0000","last_peered":"2024-04-17T06:09:26.010349+0000","last_clean":"2024-04-17T06:09:26.010349+0000","last_became_active":"2024-04-17T06:09:19.624270+0000","last_became_peered":"2024-04-17T06:09:19.624270+0000","last_unstale":"2024-04-17T06:09:26.010349+0000","last_undegraded":"2024-04-17T06:09:26.010349+0000","last_fullsized":"2024-04-17T06:09:26.010349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012635+0000","last_change":"2024-04-17T06:09:24.989917+0000","last_active":"2024-04-17T06:09:26.012635+0000","last_peered":"2024-04-17T06:09:26.012635+0000","last_clean":"2024-04-17T06:09:26.012635+0000","last_became_active":"2024-04-17T06:09:24.989308+0000","last_became_peered":"2024-04-17T06:09:24.989308+0000","last_unstale":"2024-04-17T06:09:26.012635+0000","last_undegraded":"2024-04-17T06:09:26.012635+0000","last_fullsized":"2024-04-17T06:09:26.012635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:14:19.804372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010303+0000","last_change":"2024-04-17T06:09:20.656161+0000","last_active":"2024-04-17T06:09:26.010303+0000","last_peered":"2024-04-17T06:09:26.010303+0000","last_clean":"2024-04-17T06:09:26.010303+0000","last_became_active":"2024-04-17T06:09:20.655897+0000","last_became_peered":"2024-04-17T06:09:20.655897+0000","last_unstale":"2024-04-17T06:09:26.010303+0000","last_undegraded":"2024-04-17T06:09:26.010303+0000","last_fullsized":"2024-04-17T06:09:26.010303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660641+0000","last_change":"2024-04-17T06:09:19.621223+0000","last_active":"2024-04-17T06:09:25.660641+0000","last_peered":"2024-04-17T06:09:25.660641+0000","last_clean":"2024-04-17T06:09:25.660641+0000","last_became_active":"2024-04-17T06:09:19.620409+0000","last_became_peered":"2024-04-17T06:09:19.620409+0000","last_unstale":"2024-04-17T06:09:25.660641+0000","last_undegraded":"2024-04-17T06:09:25.660641+0000","last_fullsized":"2024-04-17T06:09:25.660641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010841+0000","last_change":"2024-04-17T06:09:09.067035+0000","last_active":"2024-04-17T06:09:26.010841+0000","last_peered":"2024-04-17T06:09:26.010841+0000","last_clean":"2024-04-17T06:09:26.010841+0000","last_became_active":"2024-04-17T06:09:09.066130+0000","last_became_peered":"2024-04-17T06:09:09.066130+0000","last_unstale":"2024-04-17T06:09:26.010841+0000","last_undegraded":"2024-04-17T06:09:26.010841+0000","last_fullsized":"2024-04-17T06:09:26.010841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_clean_scrub_stamp":"2024-04-17T06:09:07.958875+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:03.277836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659861+0000","last_change":"2024-04-17T06:09:24.992008+0000","last_active":"2024-04-17T06:09:25.659861+0000","last_peered":"2024-04-17T06:09:25.659861+0000","last_clean":"2024-04-17T06:09:25.659861+0000","last_became_active":"2024-04-17T06:09:24.991448+0000","last_became_peered":"2024-04-17T06:09:24.991448+0000","last_unstale":"2024-04-17T06:09:25.659861+0000","last_undegraded":"2024-04-17T06:09:25.659861+0000","last_fullsized":"2024-04-17T06:09:25.659861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:15.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010640+0000","last_change":"2024-04-17T06:09:20.646916+0000","last_active":"2024-04-17T06:09:26.010640+0000","last_peered":"2024-04-17T06:09:26.010640+0000","last_clean":"2024-04-17T06:09:26.010640+0000","last_became_active":"2024-04-17T06:09:20.646069+0000","last_became_peered":"2024-04-17T06:09:20.646069+0000","last_unstale":"2024-04-17T06:09:26.010640+0000","last_undegraded":"2024-04-17T06:09:26.010640+0000","last_fullsized":"2024-04-17T06:09:26.010640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:11.442995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661446+0000","last_change":"2024-04-17T06:09:19.626825+0000","last_active":"2024-04-17T06:09:25.661446+0000","last_peered":"2024-04-17T06:09:25.661446+0000","last_clean":"2024-04-17T06:09:25.661446+0000","last_became_active":"2024-04-17T06:09:19.626549+0000","last_became_peered":"2024-04-17T06:09:19.626549+0000","last_unstale":"2024-04-17T06:09:25.661446+0000","last_undegraded":"2024-04-17T06:09:25.661446+0000","last_fullsized":"2024-04-17T06:09:25.661446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659329+0000","last_change":"2024-04-17T06:09:24.990665+0000","last_active":"2024-04-17T06:09:25.659329+0000","last_peered":"2024-04-17T06:09:25.659329+0000","last_clean":"2024-04-17T06:09:25.659329+0000","last_became_active":"2024-04-17T06:09:24.990262+0000","last_became_peered":"2024-04-17T06:09:24.990262+0000","last_unstale":"2024-04-17T06:09:25.659329+0000","last_undegraded":"2024-04-17T06:09:25.659329+0000","last_fullsized":"2024-04-17T06:09:25.659329+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:21.508074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660652+0000","last_change":"2024-04-17T06:09:20.650264+0000","last_active":"2024-04-17T06:09:25.660652+0000","last_peered":"2024-04-17T06:09:25.660652+0000","last_clean":"2024-04-17T06:09:25.660652+0000","last_became_active":"2024-04-17T06:09:20.649792+0000","last_became_peered":"2024-04-17T06:09:20.649792+0000","last_unstale":"2024-04-17T06:09:25.660652+0000","last_undegraded":"2024-04-17T06:09:25.660652+0000","last_fullsized":"2024-04-17T06:09:25.660652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:36.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011681+0000","last_change":"2024-04-17T06:09:19.625656+0000","last_active":"2024-04-17T06:09:26.011681+0000","last_peered":"2024-04-17T06:09:26.011681+0000","last_clean":"2024-04-17T06:09:26.011681+0000","last_became_active":"2024-04-17T06:09:19.625408+0000","last_became_peered":"2024-04-17T06:09:19.625408+0000","last_unstale":"2024-04-17T06:09:26.011681+0000","last_undegraded":"2024-04-17T06:09:26.011681+0000","last_fullsized":"2024-04-17T06:09:26.011681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011611+0000","last_change":"2024-04-17T06:09:24.684050+0000","last_active":"2024-04-17T06:09:26.011611+0000","last_peered":"2024-04-17T06:09:26.011611+0000","last_clean":"2024-04-17T06:09:26.011611+0000","last_became_active":"2024-04-17T06:09:24.683688+0000","last_became_peered":"2024-04-17T06:09:24.683688+0000","last_unstale":"2024-04-17T06:09:26.011611+0000","last_undegraded":"2024-04-17T06:09:26.011611+0000","last_fullsized":"2024-04-17T06:09:26.011611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:42:02.650317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011613+0000","last_change":"2024-04-17T06:09:20.645936+0000","last_active":"2024-04-17T06:09:26.011613+0000","last_peered":"2024-04-17T06:09:26.011613+0000","last_clean":"2024-04-17T06:09:26.011613+0000","last_became_active":"2024-04-17T06:09:20.645453+0000","last_became_peered":"2024-04-17T06:09:20.645453+0000","last_unstale":"2024-04-17T06:09:26.011613+0000","last_undegraded":"2024-04-17T06:09:26.011613+0000","last_fullsized":"2024-04-17T06:09:26.011613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:21.663057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661396+0000","last_change":"2024-04-17T06:09:19.621217+0000","last_active":"2024-04-17T06:09:25.661396+0000","last_peered":"2024-04-17T06:09:25.661396+0000","last_clean":"2024-04-17T06:09:25.661396+0000","last_became_active":"2024-04-17T06:09:19.620846+0000","last_became_peered":"2024-04-17T06:09:19.620846+0000","last_unstale":"2024-04-17T06:09:25.661396+0000","last_undegraded":"2024-04-17T06:09:25.661396+0000","last_fullsized":"2024-04-17T06:09:25.661396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659685+0000","last_change":"2024-04-17T06:09:24.991134+0000","last_active":"2024-04-17T06:09:25.659685+0000","last_peered":"2024-04-17T06:09:25.659685+0000","last_clean":"2024-04-17T06:09:25.659685+0000","last_became_active":"2024-04-17T06:09:24.990742+0000","last_became_peered":"2024-04-17T06:09:24.990742+0000","last_unstale":"2024-04-17T06:09:25.659685+0000","last_undegraded":"2024-04-17T06:09:25.659685+0000","last_fullsized":"2024-04-17T06:09:25.659685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:54.153325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011276+0000","last_change":"2024-04-17T06:09:20.645660+0000","last_active":"2024-04-17T06:09:26.011276+0000","last_peered":"2024-04-17T06:09:26.011276+0000","last_clean":"2024-04-17T06:09:26.011276+0000","last_became_active":"2024-04-17T06:09:20.645370+0000","last_became_peered":"2024-04-17T06:09:20.645370+0000","last_unstale":"2024-04-17T06:09:26.011276+0000","last_undegraded":"2024-04-17T06:09:26.011276+0000","last_fullsized":"2024-04-17T06:09:26.011276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:36.196623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011721+0000","last_change":"2024-04-17T06:09:19.633064+0000","last_active":"2024-04-17T06:09:26.011721+0000","last_peered":"2024-04-17T06:09:26.011721+0000","last_clean":"2024-04-17T06:09:26.011721+0000","last_became_active":"2024-04-17T06:09:19.632759+0000","last_became_peered":"2024-04-17T06:09:19.632759+0000","last_unstale":"2024-04-17T06:09:26.011721+0000","last_undegraded":"2024-04-17T06:09:26.011721+0000","last_fullsized":"2024-04-17T06:09:26.011721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:42:58.547080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010213+0000","last_change":"2024-04-17T06:09:24.676408+0000","last_active":"2024-04-17T06:09:26.010213+0000","last_peered":"2024-04-17T06:09:26.010213+0000","last_clean":"2024-04-17T06:09:26.010213+0000","last_became_active":"2024-04-17T06:09:24.675996+0000","last_became_peered":"2024-04-17T06:09:24.675996+0000","last_unstale":"2024-04-17T06:09:26.010213+0000","last_undegraded":"2024-04-17T06:09:26.010213+0000","last_fullsized":"2024-04-17T06:09:26.010213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:14.169938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010896+0000","last_change":"2024-04-17T06:09:20.645388+0000","last_active":"2024-04-17T06:09:26.010896+0000","last_peered":"2024-04-17T06:09:26.010896+0000","last_clean":"2024-04-17T06:09:26.010896+0000","last_became_active":"2024-04-17T06:09:20.645091+0000","last_became_peered":"2024-04-17T06:09:20.645091+0000","last_unstale":"2024-04-17T06:09:26.010896+0000","last_undegraded":"2024-04-17T06:09:26.010896+0000","last_fullsized":"2024-04-17T06:09:26.010896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660949+0000","last_change":"2024-04-17T06:09:19.624671+0000","last_active":"2024-04-17T06:09:25.660949+0000","last_peered":"2024-04-17T06:09:25.660949+0000","last_clean":"2024-04-17T06:09:25.660949+0000","last_became_active":"2024-04-17T06:09:19.624360+0000","last_became_peered":"2024-04-17T06:09:19.624360+0000","last_unstale":"2024-04-17T06:09:25.660949+0000","last_undegraded":"2024-04-17T06:09:25.660949+0000","last_fullsized":"2024-04-17T06:09:25.660949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:41:35.753743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012475+0000","last_change":"2024-04-17T06:09:24.677224+0000","last_active":"2024-04-17T06:09:26.012475+0000","last_peered":"2024-04-17T06:09:26.012475+0000","last_clean":"2024-04-17T06:09:26.012475+0000","last_became_active":"2024-04-17T06:09:24.676834+0000","last_became_peered":"2024-04-17T06:09:24.676834+0000","last_unstale":"2024-04-17T06:09:26.012475+0000","last_undegraded":"2024-04-17T06:09:26.012475+0000","last_fullsized":"2024-04-17T06:09:26.012475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:20:25.702449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659412+0000","last_change":"2024-04-17T06:09:20.640720+0000","last_active":"2024-04-17T06:09:25.659412+0000","last_peered":"2024-04-17T06:09:25.659412+0000","last_clean":"2024-04-17T06:09:25.659412+0000","last_became_active":"2024-04-17T06:09:20.639180+0000","last_became_peered":"2024-04-17T06:09:20.639180+0000","last_unstale":"2024-04-17T06:09:25.659412+0000","last_undegraded":"2024-04-17T06:09:25.659412+0000","last_fullsized":"2024-04-17T06:09:25.659412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659510+0000","last_change":"2024-04-17T06:09:19.623937+0000","last_active":"2024-04-17T06:09:25.659510+0000","last_peered":"2024-04-17T06:09:25.659510+0000","last_clean":"2024-04-17T06:09:25.659510+0000","last_became_active":"2024-04-17T06:09:19.623613+0000","last_became_peered":"2024-04-17T06:09:19.623613+0000","last_unstale":"2024-04-17T06:09:25.659510+0000","last_undegraded":"2024-04-17T06:09:25.659510+0000","last_fullsized":"2024-04-17T06:09:25.659510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010862+0000","last_change":"2024-04-17T06:09:24.991659+0000","last_active":"2024-04-17T06:09:26.010862+0000","last_peered":"2024-04-17T06:09:26.010862+0000","last_clean":"2024-04-17T06:09:26.010862+0000","last_became_active":"2024-04-17T06:09:24.991305+0000","last_became_peered":"2024-04-17T06:09:24.991305+0000","last_unstale":"2024-04-17T06:09:26.010862+0000","last_undegraded":"2024-04-17T06:09:26.010862+0000","last_fullsized":"2024-04-17T06:09:26.010862+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:33:22.763944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010997+0000","last_change":"2024-04-17T06:09:20.647491+0000","last_active":"2024-04-17T06:09:26.010997+0000","last_peered":"2024-04-17T06:09:26.010997+0000","last_clean":"2024-04-17T06:09:26.010997+0000","last_became_active":"2024-04-17T06:09:20.647222+0000","last_became_peered":"2024-04-17T06:09:20.647222+0000","last_unstale":"2024-04-17T06:09:26.010997+0000","last_undegraded":"2024-04-17T06:09:26.010997+0000","last_fullsized":"2024-04-17T06:09:26.010997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660425+0000","last_change":"2024-04-17T06:09:19.622734+0000","last_active":"2024-04-17T06:09:25.660425+0000","last_peered":"2024-04-17T06:09:25.660425+0000","last_clean":"2024-04-17T06:09:25.660425+0000","last_became_active":"2024-04-17T06:09:19.622401+0000","last_became_peered":"2024-04-17T06:09:19.622401+0000","last_unstale":"2024-04-17T06:09:25.660425+0000","last_undegraded":"2024-04-17T06:09:25.660425+0000","last_fullsized":"2024-04-17T06:09:25.660425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012343+0000","last_change":"2024-04-17T06:09:24.989713+0000","last_active":"2024-04-17T06:09:26.012343+0000","last_peered":"2024-04-17T06:09:26.012343+0000","last_clean":"2024-04-17T06:09:26.012343+0000","last_became_active":"2024-04-17T06:09:24.989311+0000","last_became_peered":"2024-04-17T06:09:24.989311+0000","last_unstale":"2024-04-17T06:09:26.012343+0000","last_undegraded":"2024-04-17T06:09:26.012343+0000","last_fullsized":"2024-04-17T06:09:26.012343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:42:09.035921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010838+0000","last_change":"2024-04-17T06:09:20.648156+0000","last_active":"2024-04-17T06:09:26.010838+0000","last_peered":"2024-04-17T06:09:26.010838+0000","last_clean":"2024-04-17T06:09:26.010838+0000","last_became_active":"2024-04-17T06:09:20.647878+0000","last_became_peered":"2024-04-17T06:09:20.647878+0000","last_unstale":"2024-04-17T06:09:26.010838+0000","last_undegraded":"2024-04-17T06:09:26.010838+0000","last_fullsized":"2024-04-17T06:09:26.010838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011779+0000","last_change":"2024-04-17T06:09:19.631943+0000","last_active":"2024-04-17T06:09:26.011779+0000","last_peered":"2024-04-17T06:09:26.011779+0000","last_clean":"2024-04-17T06:09:26.011779+0000","last_became_active":"2024-04-17T06:09:19.631287+0000","last_became_peered":"2024-04-17T06:09:19.631287+0000","last_unstale":"2024-04-17T06:09:26.011779+0000","last_undegraded":"2024-04-17T06:09:26.011779+0000","last_fullsized":"2024-04-17T06:09:26.011779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:29:44.893832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660591+0000","last_change":"2024-04-17T06:09:24.989665+0000","last_active":"2024-04-17T06:09:25.660591+0000","last_peered":"2024-04-17T06:09:25.660591+0000","last_clean":"2024-04-17T06:09:25.660591+0000","last_became_active":"2024-04-17T06:09:24.989241+0000","last_became_peered":"2024-04-17T06:09:24.989241+0000","last_unstale":"2024-04-17T06:09:25.660591+0000","last_undegraded":"2024-04-17T06:09:25.660591+0000","last_fullsized":"2024-04-17T06:09:25.660591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:27:43.835421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010532+0000","last_change":"2024-04-17T06:09:20.647147+0000","last_active":"2024-04-17T06:09:26.010532+0000","last_peered":"2024-04-17T06:09:26.010532+0000","last_clean":"2024-04-17T06:09:26.010532+0000","last_became_active":"2024-04-17T06:09:20.646657+0000","last_became_peered":"2024-04-17T06:09:20.646657+0000","last_unstale":"2024-04-17T06:09:26.010532+0000","last_undegraded":"2024-04-17T06:09:26.010532+0000","last_fullsized":"2024-04-17T06:09:26.010532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:30.596863+0000","last_change":"2024-04-17T06:09:19.629001+0000","last_active":"2024-04-17T06:09:30.596863+0000","last_peered":"2024-04-17T06:09:30.596863+0000","last_clean":"2024-04-17T06:09:30.596863+0000","last_became_active":"2024-04-17T06:09:19.628654+0000","last_became_peered":"2024-04-17T06:09:19.628654+0000","last_unstale":"2024-04-17T06:09:30.596863+0000","last_undegraded":"2024-04-17T06:09:30.596863+0000","last_fullsized":"2024-04-17T06:09:30.596863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011319+0000","last_change":"2024-04-17T06:09:24.682555+0000","last_active":"2024-04-17T06:09:26.011319+0000","last_peered":"2024-04-17T06:09:26.011319+0000","last_clean":"2024-04-17T06:09:26.011319+0000","last_became_active":"2024-04-17T06:09:24.680576+0000","last_became_peered":"2024-04-17T06:09:24.680576+0000","last_unstale":"2024-04-17T06:09:26.011319+0000","last_undegraded":"2024-04-17T06:09:26.011319+0000","last_fullsized":"2024-04-17T06:09:26.011319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:14:19.804372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010916+0000","last_change":"2024-04-17T06:09:20.644222+0000","last_active":"2024-04-17T06:09:26.010916+0000","last_peered":"2024-04-17T06:09:26.010916+0000","last_clean":"2024-04-17T06:09:26.010916+0000","last_became_active":"2024-04-17T06:09:20.643862+0000","last_became_peered":"2024-04-17T06:09:20.643862+0000","last_unstale":"2024-04-17T06:09:26.010916+0000","last_undegraded":"2024-04-17T06:09:26.010916+0000","last_fullsized":"2024-04-17T06:09:26.010916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:07.728029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011583+0000","last_change":"2024-04-17T06:09:19.628512+0000","last_active":"2024-04-17T06:09:26.011583+0000","last_peered":"2024-04-17T06:09:26.011583+0000","last_clean":"2024-04-17T06:09:26.011583+0000","last_became_active":"2024-04-17T06:09:19.628227+0000","last_became_peered":"2024-04-17T06:09:19.628227+0000","last_unstale":"2024-04-17T06:09:26.011583+0000","last_undegraded":"2024-04-17T06:09:26.011583+0000","last_fullsized":"2024-04-17T06:09:26.011583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:37:00.435911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011193+0000","last_change":"2024-04-17T06:09:24.682017+0000","last_active":"2024-04-17T06:09:26.011193+0000","last_peered":"2024-04-17T06:09:26.011193+0000","last_clean":"2024-04-17T06:09:26.011193+0000","last_became_active":"2024-04-17T06:09:24.681637+0000","last_became_peered":"2024-04-17T06:09:24.681637+0000","last_unstale":"2024-04-17T06:09:26.011193+0000","last_undegraded":"2024-04-17T06:09:26.011193+0000","last_fullsized":"2024-04-17T06:09:26.011193+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:15.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012238+0000","last_change":"2024-04-17T06:09:20.638985+0000","last_active":"2024-04-17T06:09:26.012238+0000","last_peered":"2024-04-17T06:09:26.012238+0000","last_clean":"2024-04-17T06:09:26.012238+0000","last_became_active":"2024-04-17T06:09:20.638132+0000","last_became_peered":"2024-04-17T06:09:20.638132+0000","last_unstale":"2024-04-17T06:09:26.012238+0000","last_undegraded":"2024-04-17T06:09:26.012238+0000","last_fullsized":"2024-04-17T06:09:26.012238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660523+0000","last_change":"2024-04-17T06:09:19.621789+0000","last_active":"2024-04-17T06:09:25.660523+0000","last_peered":"2024-04-17T06:09:25.660523+0000","last_clean":"2024-04-17T06:09:25.660523+0000","last_became_active":"2024-04-17T06:09:19.621506+0000","last_became_peered":"2024-04-17T06:09:19.621506+0000","last_unstale":"2024-04-17T06:09:25.660523+0000","last_undegraded":"2024-04-17T06:09:25.660523+0000","last_fullsized":"2024-04-17T06:09:25.660523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:47.976575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010264+0000","last_change":"2024-04-17T06:09:24.676641+0000","last_active":"2024-04-17T06:09:26.010264+0000","last_peered":"2024-04-17T06:09:26.010264+0000","last_clean":"2024-04-17T06:09:26.010264+0000","last_became_active":"2024-04-17T06:09:24.676251+0000","last_became_peered":"2024-04-17T06:09:24.676251+0000","last_unstale":"2024-04-17T06:09:26.010264+0000","last_undegraded":"2024-04-17T06:09:26.010264+0000","last_fullsized":"2024-04-17T06:09:26.010264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:35:54.153325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660474+0000","last_change":"2024-04-17T06:09:20.643983+0000","last_active":"2024-04-17T06:09:25.660474+0000","last_peered":"2024-04-17T06:09:25.660474+0000","last_clean":"2024-04-17T06:09:25.660474+0000","last_became_active":"2024-04-17T06:09:20.643704+0000","last_became_peered":"2024-04-17T06:09:20.643704+0000","last_unstale":"2024-04-17T06:09:25.660474+0000","last_undegraded":"2024-04-17T06:09:25.660474+0000","last_fullsized":"2024-04-17T06:09:25.660474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:19:22.149740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660308+0000","last_change":"2024-04-17T06:09:19.626344+0000","last_active":"2024-04-17T06:09:25.660308+0000","last_peered":"2024-04-17T06:09:25.660308+0000","last_clean":"2024-04-17T06:09:25.660308+0000","last_became_active":"2024-04-17T06:09:19.626060+0000","last_became_peered":"2024-04-17T06:09:19.626060+0000","last_unstale":"2024-04-17T06:09:25.660308+0000","last_undegraded":"2024-04-17T06:09:25.660308+0000","last_fullsized":"2024-04-17T06:09:25.660308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:30:52.948682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011948+0000","last_change":"2024-04-17T06:09:24.990709+0000","last_active":"2024-04-17T06:09:26.011948+0000","last_peered":"2024-04-17T06:09:26.011948+0000","last_clean":"2024-04-17T06:09:26.011948+0000","last_became_active":"2024-04-17T06:09:24.990327+0000","last_became_peered":"2024-04-17T06:09:24.990327+0000","last_unstale":"2024-04-17T06:09:26.011948+0000","last_undegraded":"2024-04-17T06:09:26.011948+0000","last_fullsized":"2024-04-17T06:09:26.011948+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:26:33.123735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012474+0000","last_change":"2024-04-17T06:09:24.994579+0000","last_active":"2024-04-17T06:09:26.012474+0000","last_peered":"2024-04-17T06:09:26.012474+0000","last_clean":"2024-04-17T06:09:26.012474+0000","last_became_active":"2024-04-17T06:09:24.994172+0000","last_became_peered":"2024-04-17T06:09:24.994172+0000","last_unstale":"2024-04-17T06:09:26.012474+0000","last_undegraded":"2024-04-17T06:09:26.012474+0000","last_fullsized":"2024-04-17T06:09:26.012474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:20.183387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010445+0000","last_change":"2024-04-17T06:09:20.646244+0000","last_active":"2024-04-17T06:09:26.010445+0000","last_peered":"2024-04-17T06:09:26.010445+0000","last_clean":"2024-04-17T06:09:26.010445+0000","last_became_active":"2024-04-17T06:09:20.645899+0000","last_became_peered":"2024-04-17T06:09:20.645899+0000","last_unstale":"2024-04-17T06:09:26.010445+0000","last_undegraded":"2024-04-17T06:09:26.010445+0000","last_fullsized":"2024-04-17T06:09:26.010445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659810+0000","last_change":"2024-04-17T06:09:19.621004+0000","last_active":"2024-04-17T06:09:25.659810+0000","last_peered":"2024-04-17T06:09:25.659810+0000","last_clean":"2024-04-17T06:09:25.659810+0000","last_became_active":"2024-04-17T06:09:19.620503+0000","last_became_peered":"2024-04-17T06:09:19.620503+0000","last_unstale":"2024-04-17T06:09:25.659810+0000","last_undegraded":"2024-04-17T06:09:25.659810+0000","last_fullsized":"2024-04-17T06:09:25.659810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660166+0000","last_change":"2024-04-17T06:09:24.674536+0000","last_active":"2024-04-17T06:09:25.660166+0000","last_peered":"2024-04-17T06:09:25.660166+0000","last_clean":"2024-04-17T06:09:25.660166+0000","last_became_active":"2024-04-17T06:09:24.674159+0000","last_became_peered":"2024-04-17T06:09:24.674159+0000","last_unstale":"2024-04-17T06:09:25.660166+0000","last_undegraded":"2024-04-17T06:09:25.660166+0000","last_fullsized":"2024-04-17T06:09:25.660166+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:18:40.236016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010314+0000","last_change":"2024-04-17T06:09:20.638332+0000","last_active":"2024-04-17T06:09:26.010314+0000","last_peered":"2024-04-17T06:09:26.010314+0000","last_clean":"2024-04-17T06:09:26.010314+0000","last_became_active":"2024-04-17T06:09:20.638096+0000","last_became_peered":"2024-04-17T06:09:20.638096+0000","last_unstale":"2024-04-17T06:09:26.010314+0000","last_undegraded":"2024-04-17T06:09:26.010314+0000","last_fullsized":"2024-04-17T06:09:26.010314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:21:10.426490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012032+0000","last_change":"2024-04-17T06:09:19.627827+0000","last_active":"2024-04-17T06:09:26.012032+0000","last_peered":"2024-04-17T06:09:26.012032+0000","last_clean":"2024-04-17T06:09:26.012032+0000","last_became_active":"2024-04-17T06:09:19.627529+0000","last_became_peered":"2024-04-17T06:09:19.627529+0000","last_unstale":"2024-04-17T06:09:26.012032+0000","last_undegraded":"2024-04-17T06:09:26.012032+0000","last_fullsized":"2024-04-17T06:09:26.012032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659711+0000","last_change":"2024-04-17T06:09:19.623047+0000","last_active":"2024-04-17T06:09:25.659711+0000","last_peered":"2024-04-17T06:09:25.659711+0000","last_clean":"2024-04-17T06:09:25.659711+0000","last_became_active":"2024-04-17T06:09:19.622700+0000","last_became_peered":"2024-04-17T06:09:19.622700+0000","last_unstale":"2024-04-17T06:09:25.659711+0000","last_undegraded":"2024-04-17T06:09:25.659711+0000","last_fullsized":"2024-04-17T06:09:25.659711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:17:00.415540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011732+0000","last_change":"2024-04-17T06:09:20.630941+0000","last_active":"2024-04-17T06:09:26.011732+0000","last_peered":"2024-04-17T06:09:26.011732+0000","last_clean":"2024-04-17T06:09:26.011732+0000","last_became_active":"2024-04-17T06:09:20.630680+0000","last_became_peered":"2024-04-17T06:09:20.630680+0000","last_unstale":"2024-04-17T06:09:26.011732+0000","last_undegraded":"2024-04-17T06:09:26.011732+0000","last_fullsized":"2024-04-17T06:09:26.011732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661370+0000","last_change":"2024-04-17T06:09:24.668919+0000","last_active":"2024-04-17T06:09:25.661370+0000","last_peered":"2024-04-17T06:09:25.661370+0000","last_clean":"2024-04-17T06:09:25.661370+0000","last_became_active":"2024-04-17T06:09:24.668456+0000","last_became_peered":"2024-04-17T06:09:24.668456+0000","last_unstale":"2024-04-17T06:09:25.661370+0000","last_undegraded":"2024-04-17T06:09:25.661370+0000","last_fullsized":"2024-04-17T06:09:25.661370+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:57.434392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011442+0000","last_change":"2024-04-17T06:09:19.622862+0000","last_active":"2024-04-17T06:09:26.011442+0000","last_peered":"2024-04-17T06:09:26.011442+0000","last_clean":"2024-04-17T06:09:26.011442+0000","last_became_active":"2024-04-17T06:09:19.622514+0000","last_became_peered":"2024-04-17T06:09:19.622514+0000","last_unstale":"2024-04-17T06:09:26.011442+0000","last_undegraded":"2024-04-17T06:09:26.011442+0000","last_fullsized":"2024-04-17T06:09:26.011442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:18.599709+0000","last_clean_scrub_stamp":"2024-04-17T06:09:18.599709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:31:46.284358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011374+0000","last_change":"2024-04-17T06:09:20.638095+0000","last_active":"2024-04-17T06:09:26.011374+0000","last_peered":"2024-04-17T06:09:26.011374+0000","last_clean":"2024-04-17T06:09:26.011374+0000","last_became_active":"2024-04-17T06:09:20.637805+0000","last_became_peered":"2024-04-17T06:09:20.637805+0000","last_unstale":"2024-04-17T06:09:26.011374+0000","last_undegraded":"2024-04-17T06:09:26.011374+0000","last_fullsized":"2024-04-17T06:09:26.011374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:48.980595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012082+0000","last_change":"2024-04-17T06:09:24.675683+0000","last_active":"2024-04-17T06:09:26.012082+0000","last_peered":"2024-04-17T06:09:26.012082+0000","last_clean":"2024-04-17T06:09:26.012082+0000","last_became_active":"2024-04-17T06:09:24.675309+0000","last_became_peered":"2024-04-17T06:09:24.675309+0000","last_unstale":"2024-04-17T06:09:26.012082+0000","last_undegraded":"2024-04-17T06:09:26.012082+0000","last_fullsized":"2024-04-17T06:09:26.012082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:23.643122+0000","last_clean_scrub_stamp":"2024-04-17T06:09:23.643122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:01:10.888244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659223+0000","last_change":"2024-04-17T06:09:20.640987+0000","last_active":"2024-04-17T06:09:25.659223+0000","last_peered":"2024-04-17T06:09:25.659223+0000","last_clean":"2024-04-17T06:09:25.659223+0000","last_became_active":"2024-04-17T06:09:20.640396+0000","last_became_peered":"2024-04-17T06:09:20.640396+0000","last_unstale":"2024-04-17T06:09:25.659223+0000","last_undegraded":"2024-04-17T06:09:25.659223+0000","last_fullsized":"2024-04-17T06:09:25.659223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660950+0000","last_change":"2024-04-17T06:09:20.642450+0000","last_active":"2024-04-17T06:09:25.660950+0000","last_peered":"2024-04-17T06:09:25.660950+0000","last_clean":"2024-04-17T06:09:25.660950+0000","last_became_active":"2024-04-17T06:09:20.639465+0000","last_became_peered":"2024-04-17T06:09:20.639465+0000","last_unstale":"2024-04-17T06:09:25.660950+0000","last_undegraded":"2024-04-17T06:09:25.660950+0000","last_fullsized":"2024-04-17T06:09:25.660950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:37:08.393904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660526+0000","last_change":"2024-04-17T06:09:20.652419+0000","last_active":"2024-04-17T06:09:25.660526+0000","last_peered":"2024-04-17T06:09:25.660526+0000","last_clean":"2024-04-17T06:09:25.660526+0000","last_became_active":"2024-04-17T06:09:20.652139+0000","last_became_peered":"2024-04-17T06:09:20.652139+0000","last_unstale":"2024-04-17T06:09:25.660526+0000","last_undegraded":"2024-04-17T06:09:25.660526+0000","last_fullsized":"2024-04-17T06:09:25.660526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:42:17.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011662+0000","last_change":"2024-04-17T06:09:20.636986+0000","last_active":"2024-04-17T06:09:26.011662+0000","last_peered":"2024-04-17T06:09:26.011662+0000","last_clean":"2024-04-17T06:09:26.011662+0000","last_became_active":"2024-04-17T06:09:20.636761+0000","last_became_peered":"2024-04-17T06:09:20.636761+0000","last_unstale":"2024-04-17T06:09:26.011662+0000","last_undegraded":"2024-04-17T06:09:26.011662+0000","last_fullsized":"2024-04-17T06:09:26.011662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:12:01.167429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.012351+0000","last_change":"2024-04-17T06:09:20.646197+0000","last_active":"2024-04-17T06:09:26.012351+0000","last_peered":"2024-04-17T06:09:26.012351+0000","last_clean":"2024-04-17T06:09:26.012351+0000","last_became_active":"2024-04-17T06:09:20.645965+0000","last_became_peered":"2024-04-17T06:09:20.645965+0000","last_unstale":"2024-04-17T06:09:26.012351+0000","last_undegraded":"2024-04-17T06:09:26.012351+0000","last_fullsized":"2024-04-17T06:09:26.012351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:28:36.520282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011415+0000","last_change":"2024-04-17T06:09:20.644696+0000","last_active":"2024-04-17T06:09:26.011415+0000","last_peered":"2024-04-17T06:09:26.011415+0000","last_clean":"2024-04-17T06:09:26.011415+0000","last_became_active":"2024-04-17T06:09:20.644322+0000","last_became_peered":"2024-04-17T06:09:20.644322+0000","last_unstale":"2024-04-17T06:09:26.011415+0000","last_undegraded":"2024-04-17T06:09:26.011415+0000","last_fullsized":"2024-04-17T06:09:26.011415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660919+0000","last_change":"2024-04-17T06:09:20.650016+0000","last_active":"2024-04-17T06:09:25.660919+0000","last_peered":"2024-04-17T06:09:25.660919+0000","last_clean":"2024-04-17T06:09:25.660919+0000","last_became_active":"2024-04-17T06:09:20.649252+0000","last_became_peered":"2024-04-17T06:09:20.649252+0000","last_unstale":"2024-04-17T06:09:25.660919+0000","last_undegraded":"2024-04-17T06:09:25.660919+0000","last_fullsized":"2024-04-17T06:09:25.660919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010916+0000","last_change":"2024-04-17T06:09:20.646003+0000","last_active":"2024-04-17T06:09:26.010916+0000","last_peered":"2024-04-17T06:09:26.010916+0000","last_clean":"2024-04-17T06:09:26.010916+0000","last_became_active":"2024-04-17T06:09:20.645732+0000","last_became_peered":"2024-04-17T06:09:20.645732+0000","last_unstale":"2024-04-17T06:09:26.010916+0000","last_undegraded":"2024-04-17T06:09:26.010916+0000","last_fullsized":"2024-04-17T06:09:26.010916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659748+0000","last_change":"2024-04-17T06:09:20.650803+0000","last_active":"2024-04-17T06:09:25.659748+0000","last_peered":"2024-04-17T06:09:25.659748+0000","last_clean":"2024-04-17T06:09:25.659748+0000","last_became_active":"2024-04-17T06:09:20.649856+0000","last_became_peered":"2024-04-17T06:09:20.649856+0000","last_unstale":"2024-04-17T06:09:25.659748+0000","last_undegraded":"2024-04-17T06:09:25.659748+0000","last_fullsized":"2024-04-17T06:09:25.659748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011942+0000","last_change":"2024-04-17T06:09:20.643474+0000","last_active":"2024-04-17T06:09:26.011942+0000","last_peered":"2024-04-17T06:09:26.011942+0000","last_clean":"2024-04-17T06:09:26.011942+0000","last_became_active":"2024-04-17T06:09:20.643125+0000","last_became_peered":"2024-04-17T06:09:20.643125+0000","last_unstale":"2024-04-17T06:09:26.011942+0000","last_undegraded":"2024-04-17T06:09:26.011942+0000","last_fullsized":"2024-04-17T06:09:26.011942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660299+0000","last_change":"2024-04-17T06:09:20.646984+0000","last_active":"2024-04-17T06:09:25.660299+0000","last_peered":"2024-04-17T06:09:25.660299+0000","last_clean":"2024-04-17T06:09:25.660299+0000","last_became_active":"2024-04-17T06:09:20.646731+0000","last_became_peered":"2024-04-17T06:09:20.646731+0000","last_unstale":"2024-04-17T06:09:25.660299+0000","last_undegraded":"2024-04-17T06:09:25.660299+0000","last_fullsized":"2024-04-17T06:09:25.660299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:23:11.442995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011435+0000","last_change":"2024-04-17T06:09:20.648582+0000","last_active":"2024-04-17T06:09:26.011435+0000","last_peered":"2024-04-17T06:09:26.011435+0000","last_clean":"2024-04-17T06:09:26.011435+0000","last_became_active":"2024-04-17T06:09:20.648345+0000","last_became_peered":"2024-04-17T06:09:20.648345+0000","last_unstale":"2024-04-17T06:09:26.011435+0000","last_undegraded":"2024-04-17T06:09:26.011435+0000","last_fullsized":"2024-04-17T06:09:26.011435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660359+0000","last_change":"2024-04-17T06:09:20.647687+0000","last_active":"2024-04-17T06:09:25.660359+0000","last_peered":"2024-04-17T06:09:25.660359+0000","last_clean":"2024-04-17T06:09:25.660359+0000","last_became_active":"2024-04-17T06:09:20.646672+0000","last_became_peered":"2024-04-17T06:09:20.646672+0000","last_unstale":"2024-04-17T06:09:25.660359+0000","last_undegraded":"2024-04-17T06:09:25.660359+0000","last_fullsized":"2024-04-17T06:09:25.660359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011526+0000","last_change":"2024-04-17T06:09:20.646327+0000","last_active":"2024-04-17T06:09:26.011526+0000","last_peered":"2024-04-17T06:09:26.011526+0000","last_clean":"2024-04-17T06:09:26.011526+0000","last_became_active":"2024-04-17T06:09:20.645768+0000","last_became_peered":"2024-04-17T06:09:20.645768+0000","last_unstale":"2024-04-17T06:09:26.011526+0000","last_undegraded":"2024-04-17T06:09:26.011526+0000","last_fullsized":"2024-04-17T06:09:26.011526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:59:55.368933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661388+0000","last_change":"2024-04-17T06:09:20.638979+0000","last_active":"2024-04-17T06:09:25.661388+0000","last_peered":"2024-04-17T06:09:25.661388+0000","last_clean":"2024-04-17T06:09:25.661388+0000","last_became_active":"2024-04-17T06:09:20.638679+0000","last_became_peered":"2024-04-17T06:09:20.638679+0000","last_unstale":"2024-04-17T06:09:25.661388+0000","last_undegraded":"2024-04-17T06:09:25.661388+0000","last_fullsized":"2024-04-17T06:09:25.661388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:08.892063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.661039+0000","last_change":"2024-04-17T06:09:20.647598+0000","last_active":"2024-04-17T06:09:25.661039+0000","last_peered":"2024-04-17T06:09:25.661039+0000","last_clean":"2024-04-17T06:09:25.661039+0000","last_became_active":"2024-04-17T06:09:20.647352+0000","last_became_peered":"2024-04-17T06:09:20.647352+0000","last_unstale":"2024-04-17T06:09:25.661039+0000","last_undegraded":"2024-04-17T06:09:25.661039+0000","last_fullsized":"2024-04-17T06:09:25.661039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:16.143995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659697+0000","last_change":"2024-04-17T06:09:20.634824+0000","last_active":"2024-04-17T06:09:25.659697+0000","last_peered":"2024-04-17T06:09:25.659697+0000","last_clean":"2024-04-17T06:09:25.659697+0000","last_became_active":"2024-04-17T06:09:20.634526+0000","last_became_peered":"2024-04-17T06:09:20.634526+0000","last_unstale":"2024-04-17T06:09:25.659697+0000","last_undegraded":"2024-04-17T06:09:25.659697+0000","last_fullsized":"2024-04-17T06:09:25.659697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010538+0000","last_change":"2024-04-17T06:09:20.643072+0000","last_active":"2024-04-17T06:09:26.010538+0000","last_peered":"2024-04-17T06:09:26.010538+0000","last_clean":"2024-04-17T06:09:26.010538+0000","last_became_active":"2024-04-17T06:09:20.642812+0000","last_became_peered":"2024-04-17T06:09:20.642812+0000","last_unstale":"2024-04-17T06:09:26.010538+0000","last_undegraded":"2024-04-17T06:09:26.010538+0000","last_fullsized":"2024-04-17T06:09:26.010538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010750+0000","last_change":"2024-04-17T06:09:20.636421+0000","last_active":"2024-04-17T06:09:26.010750+0000","last_peered":"2024-04-17T06:09:26.010750+0000","last_clean":"2024-04-17T06:09:26.010750+0000","last_became_active":"2024-04-17T06:09:20.636165+0000","last_became_peered":"2024-04-17T06:09:20.636165+0000","last_unstale":"2024-04-17T06:09:26.010750+0000","last_undegraded":"2024-04-17T06:09:26.010750+0000","last_fullsized":"2024-04-17T06:09:26.010750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:32:41.556474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011251+0000","last_change":"2024-04-17T06:09:20.639331+0000","last_active":"2024-04-17T06:09:26.011251+0000","last_peered":"2024-04-17T06:09:26.011251+0000","last_clean":"2024-04-17T06:09:26.011251+0000","last_became_active":"2024-04-17T06:09:20.639051+0000","last_became_peered":"2024-04-17T06:09:20.639051+0000","last_unstale":"2024-04-17T06:09:26.011251+0000","last_undegraded":"2024-04-17T06:09:26.011251+0000","last_fullsized":"2024-04-17T06:09:26.011251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010813+0000","last_change":"2024-04-17T06:09:20.639239+0000","last_active":"2024-04-17T06:09:26.010813+0000","last_peered":"2024-04-17T06:09:26.010813+0000","last_clean":"2024-04-17T06:09:26.010813+0000","last_became_active":"2024-04-17T06:09:20.638982+0000","last_became_peered":"2024-04-17T06:09:20.638982+0000","last_unstale":"2024-04-17T06:09:26.010813+0000","last_undegraded":"2024-04-17T06:09:26.010813+0000","last_fullsized":"2024-04-17T06:09:26.010813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:08:08.892063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.659939+0000","last_change":"2024-04-17T06:09:20.639508+0000","last_active":"2024-04-17T06:09:25.659939+0000","last_peered":"2024-04-17T06:09:25.659939+0000","last_clean":"2024-04-17T06:09:25.659939+0000","last_became_active":"2024-04-17T06:09:20.639151+0000","last_became_peered":"2024-04-17T06:09:20.639151+0000","last_unstale":"2024-04-17T06:09:25.659939+0000","last_undegraded":"2024-04-17T06:09:25.659939+0000","last_fullsized":"2024-04-17T06:09:25.659939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:34:59.711151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660890+0000","last_change":"2024-04-17T06:09:20.639084+0000","last_active":"2024-04-17T06:09:25.660890+0000","last_peered":"2024-04-17T06:09:25.660890+0000","last_clean":"2024-04-17T06:09:25.660890+0000","last_became_active":"2024-04-17T06:09:20.638793+0000","last_became_peered":"2024-04-17T06:09:20.638793+0000","last_unstale":"2024-04-17T06:09:25.660890+0000","last_undegraded":"2024-04-17T06:09:25.660890+0000","last_fullsized":"2024-04-17T06:09:25.660890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011592+0000","last_change":"2024-04-17T06:09:20.645741+0000","last_active":"2024-04-17T06:09:26.011592+0000","last_peered":"2024-04-17T06:09:26.011592+0000","last_clean":"2024-04-17T06:09:26.011592+0000","last_became_active":"2024-04-17T06:09:20.645168+0000","last_became_peered":"2024-04-17T06:09:20.645168+0000","last_unstale":"2024-04-17T06:09:26.011592+0000","last_undegraded":"2024-04-17T06:09:26.011592+0000","last_fullsized":"2024-04-17T06:09:26.011592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:16.143995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010798+0000","last_change":"2024-04-17T06:09:20.656219+0000","last_active":"2024-04-17T06:09:26.010798+0000","last_peered":"2024-04-17T06:09:26.010798+0000","last_clean":"2024-04-17T06:09:26.010798+0000","last_became_active":"2024-04-17T06:09:20.655978+0000","last_became_peered":"2024-04-17T06:09:20.655978+0000","last_unstale":"2024-04-17T06:09:26.010798+0000","last_undegraded":"2024-04-17T06:09:26.010798+0000","last_fullsized":"2024-04-17T06:09:26.010798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:10:01.455386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010761+0000","last_change":"2024-04-17T06:09:20.650090+0000","last_active":"2024-04-17T06:09:26.010761+0000","last_peered":"2024-04-17T06:09:26.010761+0000","last_clean":"2024-04-17T06:09:26.010761+0000","last_became_active":"2024-04-17T06:09:20.649596+0000","last_became_peered":"2024-04-17T06:09:20.649596+0000","last_unstale":"2024-04-17T06:09:26.010761+0000","last_undegraded":"2024-04-17T06:09:26.010761+0000","last_fullsized":"2024-04-17T06:09:26.010761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:17:17.510552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660177+0000","last_change":"2024-04-17T06:09:20.634204+0000","last_active":"2024-04-17T06:09:25.660177+0000","last_peered":"2024-04-17T06:09:25.660177+0000","last_clean":"2024-04-17T06:09:25.660177+0000","last_became_active":"2024-04-17T06:09:20.633924+0000","last_became_peered":"2024-04-17T06:09:20.633924+0000","last_unstale":"2024-04-17T06:09:25.660177+0000","last_undegraded":"2024-04-17T06:09:25.660177+0000","last_fullsized":"2024-04-17T06:09:25.660177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:10.130545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010796+0000","last_change":"2024-04-17T06:09:20.645437+0000","last_active":"2024-04-17T06:09:26.010796+0000","last_peered":"2024-04-17T06:09:26.010796+0000","last_clean":"2024-04-17T06:09:26.010796+0000","last_became_active":"2024-04-17T06:09:20.645015+0000","last_became_peered":"2024-04-17T06:09:20.645015+0000","last_unstale":"2024-04-17T06:09:26.010796+0000","last_undegraded":"2024-04-17T06:09:26.010796+0000","last_fullsized":"2024-04-17T06:09:26.010796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:25.660367+0000","last_change":"2024-04-17T06:09:20.650436+0000","last_active":"2024-04-17T06:09:25.660367+0000","last_peered":"2024-04-17T06:09:25.660367+0000","last_clean":"2024-04-17T06:09:25.660367+0000","last_became_active":"2024-04-17T06:09:20.650132+0000","last_became_peered":"2024-04-17T06:09:20.650132+0000","last_unstale":"2024-04-17T06:09:25.660367+0000","last_undegraded":"2024-04-17T06:09:25.660367+0000","last_fullsized":"2024-04-17T06:09:25.660367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:08:34.243595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011167+0000","last_change":"2024-04-17T06:09:20.634255+0000","last_active":"2024-04-17T06:09:26.011167+0000","last_peered":"2024-04-17T06:09:26.011167+0000","last_clean":"2024-04-17T06:09:26.011167+0000","last_became_active":"2024-04-17T06:09:20.633879+0000","last_became_peered":"2024-04-17T06:09:20.633879+0000","last_unstale":"2024-04-17T06:09:26.011167+0000","last_undegraded":"2024-04-17T06:09:26.011167+0000","last_fullsized":"2024-04-17T06:09:26.011167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:54:07.365672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.010938+0000","last_change":"2024-04-17T06:09:20.635023+0000","last_active":"2024-04-17T06:09:26.010938+0000","last_peered":"2024-04-17T06:09:26.010938+0000","last_clean":"2024-04-17T06:09:26.010938+0000","last_became_active":"2024-04-17T06:09:20.634245+0000","last_became_peered":"2024-04-17T06:09:20.634245+0000","last_unstale":"2024-04-17T06:09:26.010938+0000","last_undegraded":"2024-04-17T06:09:26.010938+0000","last_fullsized":"2024-04-17T06:09:26.010938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:35:35.521778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-17T06:09:26.011506+0000","last_change":"2024-04-17T06:09:20.653636+0000","last_active":"2024-04-17T06:09:26.011506+0000","last_peered":"2024-04-17T06:09:26.011506+0000","last_clean":"2024-04-17T06:09:26.011506+0000","last_became_active":"2024-04-17T06:09:20.653366+0000","last_became_peered":"2024-04-17T06:09:20.653366+0000","last_unstale":"2024-04-17T06:09:26.011506+0000","last_undegraded":"2024-04-17T06:09:26.011506+0000","last_fullsized":"2024-04-17T06:09:26.011506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:19.603729+0000","last_clean_scrub_stamp":"2024-04-17T06:09:19.603729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:41:58.610924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T06:09:42.448 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-17T06:09:42.448 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-17T06:09:42.448 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-17T06:09:42.448 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-17T06:09:42.585 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.587+0000 7f9f81d3d640 1 Processor -- start 2024-04-17T06:09:42.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.587+0000 7f9f81d3d640 1 -- start start 2024-04-17T06:09:42.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.588+0000 7f9f81d3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 0x7f9f7c15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.588+0000 7f9f81d3d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f7c0cf560 con 0x7f9f7c0b4230 2024-04-17T06:09:42.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.588+0000 7f9f81d3d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f7c160030 con 0x7f9f7c0cdac0 2024-04-17T06:09:42.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.588+0000 7f9f81d3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7c0b5400 con 0x7f9f7c159260 2024-04-17T06:09:42.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.588+0000 7f9f7affd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9f7c0b4230 0x7f9f7c0cd3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37844/0 (socket says 172.21.15.135:37844) 2024-04-17T06:09:42.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.588+0000 7f9f7affd640 1 -- 172.21.15.135:0/1141210772 learned_addr learned my addr 172.21.15.135:0/1141210772 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7b7fe640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 0x7f9f7c15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875859504 0 0) 0x7f9f70001110 con 0x7f9f7c0b4230 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f64003520 con 0x7f9f7c0b4230 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3761789701 0 0) 0x7f9f64003520 con 0x7f9f7c0b4230 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 msgr2=0x7f9f7c15faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 0x7f9f7c15faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 >> v1:172.21.15.190:6789/0 conn(0x7f9f7c0cdac0 legacy=0x7f9f7c1686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c0b5890 con 0x7f9f7c0b4230 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f9f70001840 con 0x7f9f7c0b4230 2024-04-17T06:09:42.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 >> v1:172.21.15.135:6789/0 conn(0x7f9f7c0b4230 legacy=0x7f9f7c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f64004800 0x7f9f64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 0x7f9f640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0b5400 con 0x7f9f640039e0 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6400ba70 con 0x7f9f64004800 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6400bc70 con 0x7f9f7c159260 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7b7fe640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7affd640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f64004800 0x7f9f64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7bfff640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 0x7f9f640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7b7fe640 1 -- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 msgr2=0x7f9f640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7b7fe640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 0x7f9f640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7b7fe640 1 -- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f64004800 msgr2=0x7f9f64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7b7fe640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f64004800 0x7f9f64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7b7fe640 1 -- 172.21.15.135:0/1141210772 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f64003520 con 0x7f9f640039e0 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.589+0000 7f9f7bfff640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159260 0x7f9f640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.590+0000 7f9f7b7fe640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f64003e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009f90 tx=0x7f9f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b8bce11e107e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.590+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c039070 con 0x7f9f640039e0 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.590+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c02cbd0 con 0x7f9f640039e0 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.590+0000 7f9f7a7fc640 1 -- 172.21.15.135:0/1141210772 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c009c10 con 0x7f9f640039e0 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.591+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1141210772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 msgr2=0x7f9f64003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.591+0000 7f9f81d3d640 1 --2- 172.21.15.135:0/1141210772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f64003e00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009f90 tx=0x7f9f64005830 comp rx=0 tx=0).stop 2024-04-17T06:09:42.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.591+0000 7f9f7bfff640 1 -- 172.21.15.135:0/1141210772 reap_dead start 2024-04-17T06:09:42.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.591+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1141210772 shutdown_connections 2024-04-17T06:09:42.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.591+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1141210772 >> 172.21.15.135:0/1141210772 conn(0x7f9f7c067db0 msgr2=0x7f9f7c06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:42.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.591+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1141210772 shutdown_connections 2024-04-17T06:09:42.593 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.591+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1141210772 wait complete. 2024-04-17T06:09:42.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.592+0000 7f9f81d3d640 1 Processor -- start 2024-04-17T06:09:42.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.592+0000 7f9f81d3d640 1 -- start start 2024-04-17T06:09:42.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f81d3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f7c0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f81d3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159640 0x7f9f7c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f81d3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b6e0 0x7f9f7c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f81d3d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0b4db0 con 0x7f9f640039e0 2024-04-17T06:09:42.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f81d3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0b4ad0 con 0x7f9f7c15b6e0 2024-04-17T06:09:42.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f81d3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7c0b5090 con 0x7f9f7c159640 2024-04-17T06:09:42.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f7c0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f7c0affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49054/0 (socket says 172.21.15.135:49054) 2024-04-17T06:09:42.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 -- 172.21.15.135:0/1054474814 learned_addr learned my addr 172.21.15.135:0/1054474814 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:42.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7bfff640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b6e0 0x7f9f7c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7affd640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159640 0x7f9f7c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 -- 172.21.15.135:0/1054474814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159640 msgr2=0x7f9f7c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159640 0x7f9f7c15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 -- 172.21.15.135:0/1054474814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b6e0 msgr2=0x7f9f7c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b6e0 0x7f9f7c1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 -- 172.21.15.135:0/1054474814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c15c020 con 0x7f9f640039e0 2024-04-17T06:09:42.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.593+0000 7f9f7b7fe640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f7c0affa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c02c640 tx=0x7f9f6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.602 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.594+0000 7f9f63fff640 1 -- 172.21.15.135:0/1054474814 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c042020 con 0x7f9f640039e0 2024-04-17T06:09:42.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.594+0000 7f9f63fff640 1 -- 172.21.15.135:0/1054474814 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c03a040 con 0x7f9f640039e0 2024-04-17T06:09:42.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.594+0000 7f9f63fff640 1 -- 172.21.15.135:0/1054474814 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c039070 con 0x7f9f640039e0 2024-04-17T06:09:42.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.594+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c15c300 con 0x7f9f640039e0 2024-04-17T06:09:42.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.594+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c145670 con 0x7f9f640039e0 2024-04-17T06:09:42.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.595+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c074940 con 0x7f9f640039e0 2024-04-17T06:09:42.603 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.595+0000 7f9f63fff640 1 -- 172.21.15.135:0/1054474814 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9f6c005d10 con 0x7f9f640039e0 2024-04-17T06:09:42.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.595+0000 7f9f63fff640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.596+0000 7f9f63fff640 1 -- 172.21.15.135:0/1054474814 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9f6c0ae320 con 0x7f9f640039e0 2024-04-17T06:09:42.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.596+0000 7f9f7affd640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.596+0000 7f9f7affd640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f6806c8f0 0x7f9f6806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c06eb00 tx=0x7f9f70001320 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.600+0000 7f9f63fff640 1 -- 172.21.15.135:0/1054474814 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f6c04c030 con 0x7f9f640039e0 2024-04-17T06:09:42.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.811+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9f7c0ad860 con 0x7f9f640039e0 2024-04-17T06:09:42.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.811+0000 7f9f63fff640 1 -- 172.21.15.135:0/1054474814 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9f6c07c440 con 0x7f9f640039e0 2024-04-17T06:09:42.809 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:42.809 INFO:teuthology.orchestra.run.smithi135.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f6806c8f0 msgr2=0x7f9f6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f6806c8f0 0x7f9f6806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c06eb00 tx=0x7f9f70001320 comp rx=0 tx=0).stop 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 msgr2=0x7f9f7c0affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f7c0affa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c02c640 tx=0x7f9f6c009750 comp rx=0 tx=0).stop 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 shutdown_connections 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b6e0 0x7f9f7c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c159640 0x7f9f7c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9f6806c8f0 0x7f9f6806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 --2- 172.21.15.135:0/1054474814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f640039e0 0x7f9f7c0affa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 >> 172.21.15.135:0/1054474814 conn(0x7f9f7c067db0 msgr2=0x7f9f7c152360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 shutdown_connections 2024-04-17T06:09:42.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.813+0000 7f9f81d3d640 1 -- 172.21.15.135:0/1054474814 wait complete. 2024-04-17T06:09:42.822 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-17T06:09:42.823 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:42.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.927+0000 7fd15004e640 1 Processor -- start 2024-04-17T06:09:42.926 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.927+0000 7fd15004e640 1 -- start start 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.928+0000 7fd15004e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd148144f70 0x7fd1480cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.928+0000 7fd15004e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd148073750 0x7fd14815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.928+0000 7fd15004e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1480b75c0 con 0x7fd14815f500 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.928+0000 7fd15004e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd1480cf130 con 0x7fd148144f70 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.928+0000 7fd15004e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd148160070 con 0x7fd148073750 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.929+0000 7fd14ddea640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd148144f70 0x7fd1480cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.929+0000 7fd14ddea640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd148144f70 0x7fd1480cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49064/0 (socket says 172.21.15.135:49064) 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.929+0000 7fd14ddea640 1 -- 172.21.15.135:0/2942013875 learned_addr learned my addr 172.21.15.135:0/2942013875 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.929+0000 7fd14ddea640 1 -- 172.21.15.135:0/2942013875 >> v2:172.21.15.190:3300/0 conn(0x7fd148073750 msgr2=0x7fd14815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.929+0000 7fd14ddea640 1 --2- 172.21.15.135:0/2942013875 >> v2:172.21.15.190:3300/0 conn(0x7fd148073750 0x7fd14815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.929+0000 7fd14ddea640 1 -- 172.21.15.135:0/2942013875 >> v1:172.21.15.190:6789/0 conn(0x7fd14815f500 legacy=0x7fd148073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:42.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.929+0000 7fd14ddea640 1 -- 172.21.15.135:0/2942013875 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd148160350 con 0x7fd148144f70 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd14ddea640 1 --2- 172.21.15.135:0/2942013875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148144f70 0x7fd1480cebf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1480b55b0 tx=0x7fd14402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa1852e0bda1cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd14cde8640 1 -- 172.21.15.135:0/2942013875 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14402c9f0 con 0x7fd148144f70 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd14cde8640 1 -- 172.21.15.135:0/2942013875 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14402cb90 con 0x7fd148144f70 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd14cde8640 1 -- 172.21.15.135:0/2942013875 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440343f0 con 0x7fd148144f70 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd15004e640 1 -- 172.21.15.135:0/2942013875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148144f70 msgr2=0x7fd1480cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd15004e640 1 --2- 172.21.15.135:0/2942013875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148144f70 0x7fd1480cebf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd1480b55b0 tx=0x7fd14402be50 comp rx=0 tx=0).stop 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd15004e640 1 -- 172.21.15.135:0/2942013875 shutdown_connections 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd15004e640 1 --2- 172.21.15.135:0/2942013875 >> v2:172.21.15.190:3300/0 conn(0x7fd148073750 0x7fd14815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd15004e640 1 --2- 172.21.15.135:0/2942013875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148144f70 0x7fd1480cebf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd15004e640 1 -- 172.21.15.135:0/2942013875 >> 172.21.15.135:0/2942013875 conn(0x7fd1480d15e0 msgr2=0x7fd1480d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:42.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd15004e640 1 -- 172.21.15.135:0/2942013875 shutdown_connections 2024-04-17T06:09:42.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.930+0000 7fd15004e640 1 -- 172.21.15.135:0/2942013875 wait complete. 2024-04-17T06:09:42.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd15004e640 1 Processor -- start 2024-04-17T06:09:42.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd15004e640 1 -- start start 2024-04-17T06:09:42.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd15004e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148073750 0x7fd148156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd15004e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd148144f70 0x7fd1480d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd15004e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14815f500 0x7fd1480d0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd15004e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd148145350 con 0x7fd148073750 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd15004e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd1480ce3f0 con 0x7fd14815f500 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd15004e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd1481454d0 con 0x7fd148144f70 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.932+0000 7fd14ddea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148073750 0x7fd148156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14ddea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148073750 0x7fd148156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49074/0 (socket says 172.21.15.135:49074) 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14ddea640 1 -- 172.21.15.135:0/1614008628 learned_addr learned my addr 172.21.15.135:0/1614008628 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14d5e9640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd148144f70 0x7fd1480d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14e5eb640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14815f500 0x7fd1480d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14ddea640 1 -- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd148144f70 msgr2=0x7fd1480d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14ddea640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd148144f70 0x7fd1480d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14ddea640 1 -- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14815f500 msgr2=0x7fd1480d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14ddea640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14815f500 0x7fd1480d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14ddea640 1 -- 172.21.15.135:0/1614008628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd148157630 con 0x7fd148073750 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14e5eb640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14815f500 0x7fd1480d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd14ddea640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148073750 0x7fd148156ca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd1440093e0 tx=0x7fd144004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd13e7fc640 1 -- 172.21.15.135:0/1614008628 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144043020 con 0x7fd148073750 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd13e7fc640 1 -- 172.21.15.135:0/1614008628 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd14403e450 con 0x7fd148073750 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd13e7fc640 1 -- 172.21.15.135:0/1614008628 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14403e840 con 0x7fd148073750 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd148157910 con 0x7fd148073750 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.933+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd148157ed0 con 0x7fd148073750 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.934+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd148068a20 con 0x7fd148073750 2024-04-17T06:09:42.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.935+0000 7fd13e7fc640 1 -- 172.21.15.135:0/1614008628 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd144046020 con 0x7fd148073750 2024-04-17T06:09:42.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.935+0000 7fd13e7fc640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12c06c8f0 0x7fd12c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:42.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.935+0000 7fd13e7fc640 1 -- 172.21.15.135:0/1614008628 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd1440ae850 con 0x7fd148073750 2024-04-17T06:09:42.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.935+0000 7fd14d5e9640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12c06c8f0 0x7fd12c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:42.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.935+0000 7fd14d5e9640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12c06c8f0 0x7fd12c06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd1481f61c0 tx=0x7fd138009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:42.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:42.939+0000 7fd13e7fc640 1 -- 172.21.15.135:0/1614008628 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd14404d030 con 0x7fd148073750 2024-04-17T06:09:43.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.116+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd148157410 con 0x7fd148073750 2024-04-17T06:09:43.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.117+0000 7fd13e7fc640 1 -- 172.21.15.135:0/1614008628 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7fd14407c9d0 con 0x7fd148073750 2024-04-17T06:09:43.115 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:43.116 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:43.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12c06c8f0 msgr2=0x7fd12c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12c06c8f0 0x7fd12c06edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd1481f61c0 tx=0x7fd138009040 comp rx=0 tx=0).stop 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148073750 msgr2=0x7fd148156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148073750 0x7fd148156ca0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd1440093e0 tx=0x7fd144004a40 comp rx=0 tx=0).stop 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 shutdown_connections 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd14815f500 0x7fd1480d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd148144f70 0x7fd1480d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12c06c8f0 0x7fd12c06edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 --2- 172.21.15.135:0/1614008628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd148073750 0x7fd148156ca0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 >> 172.21.15.135:0/1614008628 conn(0x7fd1480d15e0 msgr2=0x7fd1480b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 shutdown_connections 2024-04-17T06:09:43.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.119+0000 7fd15004e640 1 -- 172.21.15.135:0/1614008628 wait complete. 2024-04-17T06:09:43.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:09:43.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:43.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.237+0000 7f091e0a7640 1 Processor -- start 2024-04-17T06:09:43.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.237+0000 7f091e0a7640 1 -- start start 2024-04-17T06:09:43.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f091e0a7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f091815cac0 0x7f091806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f091e0a7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f091806d020 0x7f0918070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f091e0a7640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f091815cea0 con 0x7f09180d03d0 2024-04-17T06:09:43.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f091e0a7640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f09180d0940 con 0x7f091806d020 2024-04-17T06:09:43.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f091e0a7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f091806c540 con 0x7f091815cac0 2024-04-17T06:09:43.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f0916ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f091806d020 0x7f0918070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f0917fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f09180d03d0 0x7f09180cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37848/0 (socket says 172.21.15.135:37848) 2024-04-17T06:09:43.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f0917fff640 1 -- 172.21.15.135:0/3490082401 learned_addr learned my addr 172.21.15.135:0/3490082401 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f09177fe640 1 --2- 172.21.15.135:0/3490082401 >> v2:172.21.15.190:3300/0 conn(0x7f091815cac0 0x7f091806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f0916ffd640 1 -- 172.21.15.135:0/3490082401 >> v2:172.21.15.190:3300/0 conn(0x7f091815cac0 msgr2=0x7f091806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f0916ffd640 1 --2- 172.21.15.135:0/3490082401 >> v2:172.21.15.190:3300/0 conn(0x7f091815cac0 0x7f091806e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f0916ffd640 1 -- 172.21.15.135:0/3490082401 >> v1:172.21.15.135:6789/0 conn(0x7f09180d03d0 legacy=0x7f09180cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f0916ffd640 1 -- 172.21.15.135:0/3490082401 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091806c820 con 0x7f091806d020 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.238+0000 7f09167fc640 1 -- 172.21.15.135:0/3490082401 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 874099151 0 0) 0x7f091815cea0 con 0x7f09180d03d0 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.239+0000 7f0916ffd640 1 --2- 172.21.15.135:0/3490082401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918070370 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f091806d560 tx=0x7f090c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60460946aaff6ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.239+0000 7f09167fc640 1 -- 172.21.15.135:0/3490082401 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c035020 con 0x7f091806d020 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.239+0000 7f09167fc640 1 -- 172.21.15.135:0/3490082401 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090c031830 con 0x7f091806d020 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.239+0000 7f09167fc640 1 -- 172.21.15.135:0/3490082401 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c031c00 con 0x7f091806d020 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.240+0000 7f091e0a7640 1 -- 172.21.15.135:0/3490082401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 msgr2=0x7f0918070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.240+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3490082401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918070370 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f091806d560 tx=0x7f090c02b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.240+0000 7f091e0a7640 1 -- 172.21.15.135:0/3490082401 shutdown_connections 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.240+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3490082401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918070370 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.240+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3490082401 >> v2:172.21.15.190:3300/0 conn(0x7f091815cac0 0x7f091806e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.240+0000 7f091e0a7640 1 -- 172.21.15.135:0/3490082401 >> 172.21.15.135:0/3490082401 conn(0x7f09180cd7e0 msgr2=0x7f09180cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:43.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.240+0000 7f091e0a7640 1 -- 172.21.15.135:0/3490082401 shutdown_connections 2024-04-17T06:09:43.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.240+0000 7f091e0a7640 1 -- 172.21.15.135:0/3490082401 wait complete. 2024-04-17T06:09:43.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.241+0000 7f091e0a7640 1 Processor -- start 2024-04-17T06:09:43.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.241+0000 7f091e0a7640 1 -- start start 2024-04-17T06:09:43.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f091e0a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f091e0a7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09180d03d0 0x7f09181493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f091e0a7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f091815cac0 0x7f09181e3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f091e0a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f091806ef10 con 0x7f091806d020 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f091e0a7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f091806ed90 con 0x7f09180d03d0 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f091e0a7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f09180708b0 con 0x7f091815cac0 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918146150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49092/0 (socket says 172.21.15.135:49092) 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 -- 172.21.15.135:0/3781161066 learned_addr learned my addr 172.21.15.135:0/3781161066 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 -- 172.21.15.135:0/3781161066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f091815cac0 msgr2=0x7f09181e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f091815cac0 0x7f09181e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 -- 172.21.15.135:0/3781161066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09180d03d0 msgr2=0x7f09181493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09180d03d0 0x7f09181493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 -- 172.21.15.135:0/3781161066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09181e5ea0 con 0x7f091806d020 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.242+0000 7f09177fe640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918146150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0908007e40 tx=0x7f090800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.243+0000 7f08fffff640 1 -- 172.21.15.135:0/3781161066 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908014620 con 0x7f091806d020 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.243+0000 7f08fffff640 1 -- 172.21.15.135:0/3781161066 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0908019070 con 0x7f091806d020 2024-04-17T06:09:43.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.243+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0918146690 con 0x7f091806d020 2024-04-17T06:09:43.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.243+0000 7f08fffff640 1 -- 172.21.15.135:0/3781161066 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908014a00 con 0x7f091806d020 2024-04-17T06:09:43.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.243+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0918147140 con 0x7f091806d020 2024-04-17T06:09:43.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.244+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0918074940 con 0x7f091806d020 2024-04-17T06:09:43.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.244+0000 7f08fffff640 1 -- 172.21.15.135:0/3781161066 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0908014ba0 con 0x7f091806d020 2024-04-17T06:09:43.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.245+0000 7f08fffff640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.245+0000 7f0916ffd640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.245+0000 7f08fffff640 1 -- 172.21.15.135:0/3781161066 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f090808c6a0 con 0x7f091806d020 2024-04-17T06:09:43.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.245+0000 7f0916ffd640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f090406c8f0 0x7f090406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f090c036000 tx=0x7f090c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.249+0000 7f08fffff640 1 -- 172.21.15.135:0/3781161066 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f090805a820 con 0x7f091806d020 2024-04-17T06:09:43.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.423+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f091815ae80 con 0x7f091806d020 2024-04-17T06:09:43.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.424+0000 7f08fffff640 1 -- 172.21.15.135:0/3781161066 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f090805a120 con 0x7f091806d020 2024-04-17T06:09:43.422 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:43.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f090406c8f0 msgr2=0x7f090406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f090406c8f0 0x7f090406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f090c036000 tx=0x7f090c030000 comp rx=0 tx=0).stop 2024-04-17T06:09:43.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 msgr2=0x7f0918146150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918146150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0908007e40 tx=0x7f090800e4b0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 shutdown_connections 2024-04-17T06:09:43.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f091815cac0 0x7f09181e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09180d03d0 0x7f09181493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f090406c8f0 0x7f090406edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 --2- 172.21.15.135:0/3781161066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f091806d020 0x7f0918146150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.425+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 >> 172.21.15.135:0/3781161066 conn(0x7f09180cd7e0 msgr2=0x7f09180af440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:43.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.426+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 shutdown_connections 2024-04-17T06:09:43.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.426+0000 7f091e0a7640 1 -- 172.21.15.135:0/3781161066 wait complete. 2024-04-17T06:09:43.435 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:09:43.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.543+0000 7f0850e37640 1 Processor -- start 2024-04-17T06:09:43.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.543+0000 7f0850e37640 1 -- start start 2024-04-17T06:09:43.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f0850e37640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f084c0cfe10 0x7f084c169b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f0850e37640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f084c160060 con 0x7f084c06b990 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f0850e37640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f084c0b5ae0 con 0x7f084c15ec70 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f0850e37640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f084c06bdf0 con 0x7f084c0cfe10 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f084ad9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f084c0cfe10 0x7f084c169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f0843fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f084c06b990 0x7f084c0cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37852/0 (socket says 172.21.15.135:37852) 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f0843fff640 1 -- 172.21.15.135:0/276295090 learned_addr learned my addr 172.21.15.135:0/276295090 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f084ad9d640 1 -- 172.21.15.135:0/276295090 >> v1:172.21.15.190:6789/0 conn(0x7f084c15ec70 legacy=0x7f084c0d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f084ad9d640 1 -- 172.21.15.135:0/276295090 >> v1:172.21.15.135:6789/0 conn(0x7f084c06b990 legacy=0x7f084c0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f084ad9d640 1 -- 172.21.15.135:0/276295090 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c0b5dc0 con 0x7f084c0cfe10 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.544+0000 7f0849d9b640 1 -- 172.21.15.135:0/276295090 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2987920744 0 0) 0x7f084c160060 con 0x7f084c06b990 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f084ad9d640 1 --2- 172.21.15.135:0/276295090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c0cfe10 0x7f084c169b20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f084c0ce520 tx=0x7f084402be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a587553699bc3c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0849d9b640 1 -- 172.21.15.135:0/276295090 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844039070 con 0x7f084c0cfe10 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0849d9b640 1 -- 172.21.15.135:0/276295090 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084402cbb0 con 0x7f084c0cfe10 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0849d9b640 1 -- 172.21.15.135:0/276295090 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844034440 con 0x7f084c0cfe10 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0850e37640 1 -- 172.21.15.135:0/276295090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c0cfe10 msgr2=0x7f084c169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0850e37640 1 --2- 172.21.15.135:0/276295090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c0cfe10 0x7f084c169b20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f084c0ce520 tx=0x7f084402be00 comp rx=0 tx=0).stop 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0850e37640 1 -- 172.21.15.135:0/276295090 shutdown_connections 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0850e37640 1 --2- 172.21.15.135:0/276295090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c0cfe10 0x7f084c169b20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0850e37640 1 -- 172.21.15.135:0/276295090 >> 172.21.15.135:0/276295090 conn(0x7f084c0cd7e0 msgr2=0x7f084c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.545+0000 7f0850e37640 1 -- 172.21.15.135:0/276295090 shutdown_connections 2024-04-17T06:09:43.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.546+0000 7f0850e37640 1 -- 172.21.15.135:0/276295090 wait complete. 2024-04-17T06:09:43.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.547+0000 7f0850e37640 1 Processor -- start 2024-04-17T06:09:43.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.547+0000 7f0850e37640 1 -- start start 2024-04-17T06:09:43.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0850e37640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c06b990 0x7f084c159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0850e37640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f084c0cfe10 0x7f084c149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0850e37640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c15ec70 0x7f084c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0850e37640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f084c0b4160 con 0x7f084c15ec70 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0850e37640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f084c0b60a0 con 0x7f084c06b990 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0850e37640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f084c0d0240 con 0x7f084c0cfe10 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c15ec70 0x7f084c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c15ec70 0x7f084c06f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49108/0 (socket says 172.21.15.135:49108) 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 -- 172.21.15.135:0/3602374873 learned_addr learned my addr 172.21.15.135:0/3602374873 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0843fff640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f084c0cfe10 0x7f084c149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084a59c640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c06b990 0x7f084c159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 -- 172.21.15.135:0/3602374873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f084c0cfe10 msgr2=0x7f084c149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f084c0cfe10 0x7f084c149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 -- 172.21.15.135:0/3602374873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c06b990 msgr2=0x7f084c159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c06b990 0x7f084c159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 -- 172.21.15.135:0/3602374873 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c1f8140 con 0x7f084c15ec70 2024-04-17T06:09:43.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f084ad9d640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c15ec70 0x7f084c06f830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f084402c410 tx=0x7f0844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0842ffd640 1 -- 172.21.15.135:0/3602374873 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844044020 con 0x7f084c15ec70 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0842ffd640 1 -- 172.21.15.135:0/3602374873 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084403f470 con 0x7f084c15ec70 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0842ffd640 1 -- 172.21.15.135:0/3602374873 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844039040 con 0x7f084c15ec70 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c1f8420 con 0x7f084c15ec70 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.548+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1f89e0 con 0x7f084c15ec70 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.549+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c074940 con 0x7f084c15ec70 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.550+0000 7f0842ffd640 1 -- 172.21.15.135:0/3602374873 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0844047020 con 0x7f084c15ec70 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.550+0000 7f0842ffd640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f082c06c920 0x7f082c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.550+0000 7f0842ffd640 1 -- 172.21.15.135:0/3602374873 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f08440ae6d0 con 0x7f084c15ec70 2024-04-17T06:09:43.550 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.550+0000 7f084a59c640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f082c06c920 0x7f082c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.551+0000 7f084a59c640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f082c06c920 0x7f082c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f084c149f80 tx=0x7f083c009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.555+0000 7f0842ffd640 1 -- 172.21.15.135:0/3602374873 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f084404e070 con 0x7f084c15ec70 2024-04-17T06:09:43.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.731+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f084c0ce810 con 0x7f084c15ec70 2024-04-17T06:09:43.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.731+0000 7f0842ffd640 1 -- 172.21.15.135:0/3602374873 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f084407c170 con 0x7f084c15ec70 2024-04-17T06:09:43.729 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:43.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f082c06c920 msgr2=0x7f082c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f082c06c920 0x7f082c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f084c149f80 tx=0x7f083c009060 comp rx=0 tx=0).stop 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c15ec70 msgr2=0x7f084c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c15ec70 0x7f084c06f830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f084402c410 tx=0x7f0844004a40 comp rx=0 tx=0).stop 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 shutdown_connections 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f082c06c920 0x7f082c06ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f084c15ec70 0x7f084c06f830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f084c0cfe10 0x7f084c149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 --2- 172.21.15.135:0/3602374873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f084c06b990 0x7f084c159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 >> 172.21.15.135:0/3602374873 conn(0x7f084c0cd7e0 msgr2=0x7f084c06ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.733+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 shutdown_connections 2024-04-17T06:09:43.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.734+0000 7f0850e37640 1 -- 172.21.15.135:0/3602374873 wait complete. 2024-04-17T06:09:43.743 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:09:43.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.850+0000 7fceb76a0640 1 Processor -- start 2024-04-17T06:09:43.849 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.851+0000 7fceb76a0640 1 -- start start 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.851+0000 7fceb76a0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.851+0000 7fceb76a0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fceb00b0da0 0x7fceb006b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.851+0000 7fceb76a0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceb015f010 con 0x7fceb006bed0 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.851+0000 7fceb76a0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fceb00d0330 con 0x7fceb00b0da0 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.851+0000 7fceb76a0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fceb0073040 con 0x7fceb015ec30 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.852+0000 7fceb4c3b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fceb00b0da0 0x7fceb006b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.852+0000 7fceb543c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.852+0000 7fceb543c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb00b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:50224/0 (socket says 172.21.15.135:50224) 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.852+0000 7fceb543c640 1 -- 172.21.15.135:0/1965281427 learned_addr learned my addr 172.21.15.135:0/1965281427 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:43.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.852+0000 7fceb543c640 1 -- 172.21.15.135:0/1965281427 >> v2:172.21.15.190:3300/0 conn(0x7fceb00b0da0 msgr2=0x7fceb006b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:43.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.852+0000 7fceb543c640 1 --2- 172.21.15.135:0/1965281427 >> v2:172.21.15.190:3300/0 conn(0x7fceb00b0da0 0x7fceb006b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.852+0000 7fceb543c640 1 -- 172.21.15.135:0/1965281427 >> v1:172.21.15.190:6789/0 conn(0x7fceb006bed0 legacy=0x7fceb00708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.852+0000 7fceb543c640 1 -- 172.21.15.135:0/1965281427 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb0073290 con 0x7fceb015ec30 2024-04-17T06:09:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.853+0000 7fceb543c640 1 --2- 172.21.15.135:0/1965281427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb00b0860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009e50 tx=0x7fcea002bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e32399e4e021a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.853+0000 7fceaffff640 1 -- 172.21.15.135:0/1965281427 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea002cb90 con 0x7fceb015ec30 2024-04-17T06:09:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.853+0000 7fceaffff640 1 -- 172.21.15.135:0/1965281427 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea002cd30 con 0x7fceb015ec30 2024-04-17T06:09:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.853+0000 7fceaffff640 1 -- 172.21.15.135:0/1965281427 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0034610 con 0x7fceb015ec30 2024-04-17T06:09:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.854+0000 7fceb76a0640 1 -- 172.21.15.135:0/1965281427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 msgr2=0x7fceb00b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.854+0000 7fceb76a0640 1 --2- 172.21.15.135:0/1965281427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb00b0860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009e50 tx=0x7fcea002bff0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.854+0000 7fceb76a0640 1 -- 172.21.15.135:0/1965281427 shutdown_connections 2024-04-17T06:09:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.854+0000 7fceb76a0640 1 --2- 172.21.15.135:0/1965281427 >> v2:172.21.15.190:3300/0 conn(0x7fceb00b0da0 0x7fceb006b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.854+0000 7fceb76a0640 1 --2- 172.21.15.135:0/1965281427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb00b0860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.854+0000 7fceb76a0640 1 -- 172.21.15.135:0/1965281427 >> 172.21.15.135:0/1965281427 conn(0x7fceb00d15e0 msgr2=0x7fceb00d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.854+0000 7fceb76a0640 1 -- 172.21.15.135:0/1965281427 shutdown_connections 2024-04-17T06:09:43.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.854+0000 7fceb76a0640 1 -- 172.21.15.135:0/1965281427 wait complete. 2024-04-17T06:09:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.855+0000 7fceb76a0640 1 Processor -- start 2024-04-17T06:09:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.855+0000 7fceb76a0640 1 -- start start 2024-04-17T06:09:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.855+0000 7fceb76a0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fceb006bed0 0x7fceb014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb76a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb00b0da0 0x7fceb00caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb76a0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb0160270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb76a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0073850 con 0x7fceb00b0da0 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb76a0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fceb006c300 con 0x7fceb006bed0 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb76a0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fceb0073b30 con 0x7fceb015ec30 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb00b0da0 0x7fceb00caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb00b0da0 0x7fceb00caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49124/0 (socket says 172.21.15.135:49124) 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 -- 172.21.15.135:0/3308394267 learned_addr learned my addr 172.21.15.135:0/3308394267 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb5c3d640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb0160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb543c640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fceb006bed0 0x7fceb014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 -- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 msgr2=0x7fceb0160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb0160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 -- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fceb006bed0 msgr2=0x7fceb014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fceb006bed0 0x7fceb014e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:43.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 -- 172.21.15.135:0/3308394267 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb00cd8b0 con 0x7fceb00b0da0 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb5c3d640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb0160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb543c640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fceb006bed0 0x7fceb014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb4c3b640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb00b0da0 0x7fceb00caa10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcea400e9b0 tx=0x7fcea400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceadffb640 1 -- 172.21.15.135:0/3308394267 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4017070 con 0x7fceb00b0da0 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceadffb640 1 -- 172.21.15.135:0/3308394267 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea400c9a0 con 0x7fceb00b0da0 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb00cdb90 con 0x7fceb00b0da0 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.856+0000 7fceadffb640 1 -- 172.21.15.135:0/3308394267 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea400cd90 con 0x7fceb00b0da0 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.857+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb00ce150 con 0x7fceb00b0da0 2024-04-17T06:09:43.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.858+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb0068a20 con 0x7fceb00b0da0 2024-04-17T06:09:43.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.858+0000 7fceadffb640 1 -- 172.21.15.135:0/3308394267 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcea401a070 con 0x7fceb00b0da0 2024-04-17T06:09:43.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.858+0000 7fceadffb640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce9c06c8f0 0x7fce9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:43.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.859+0000 7fceadffb640 1 -- 172.21.15.135:0/3308394267 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcea408bea0 con 0x7fceb00b0da0 2024-04-17T06:09:43.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.859+0000 7fceb543c640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce9c06c8f0 0x7fce9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:43.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.859+0000 7fceb543c640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce9c06c8f0 0x7fce9c06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcea0035000 tx=0x7fcea0030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:43.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:43.863+0000 7fceadffb640 1 -- 172.21.15.135:0/3308394267 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcea4059f90 con 0x7fceb00b0da0 2024-04-17T06:09:44.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.038+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fceb0159410 con 0x7fceb00b0da0 2024-04-17T06:09:44.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.039+0000 7fceadffb640 1 -- 172.21.15.135:0/3308394267 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fcea4059890 con 0x7fceb00b0da0 2024-04-17T06:09:44.037 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:09:44.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce9c06c8f0 msgr2=0x7fce9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce9c06c8f0 0x7fce9c06edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcea0035000 tx=0x7fcea0030000 comp rx=0 tx=0).stop 2024-04-17T06:09:44.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb00b0da0 msgr2=0x7fceb00caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb00b0da0 0x7fceb00caa10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcea400e9b0 tx=0x7fcea400ee70 comp rx=0 tx=0).stop 2024-04-17T06:09:44.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 shutdown_connections 2024-04-17T06:09:44.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb015ec30 0x7fceb0160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fce9c06c8f0 0x7fce9c06edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb00b0da0 0x7fceb00caa10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 --2- 172.21.15.135:0/3308394267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fceb006bed0 0x7fceb014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 >> 172.21.15.135:0/3308394267 conn(0x7fceb00d15e0 msgr2=0x7fceb00cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:44.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 shutdown_connections 2024-04-17T06:09:44.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.041+0000 7fceb76a0640 1 -- 172.21.15.135:0/3308394267 wait complete. 2024-04-17T06:09:44.050 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:09:44.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.159+0000 7fdf45eba640 1 Processor -- start 2024-04-17T06:09:44.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.159+0000 7fdf45eba640 1 -- start start 2024-04-17T06:09:44.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf45eba640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdf4015ebb0 0x7fdf400d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf45eba640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf400d3230 con 0x7fdf400ba560 2024-04-17T06:09:44.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf45eba640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf400d3510 con 0x7fdf400d1850 2024-04-17T06:09:44.168 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf45eba640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdf400d37f0 con 0x7fdf4015ebb0 2024-04-17T06:09:44.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3effd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fdf400ba560 0x7fdf400d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37864/0 (socket says 172.21.15.135:37864) 2024-04-17T06:09:44.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3effd640 1 -- 172.21.15.135:0/1693451757 learned_addr learned my addr 172.21.15.135:0/1693451757 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:44.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3f7fe640 1 --2- 172.21.15.135:0/1693451757 >> v2:172.21.15.190:3300/0 conn(0x7fdf4015ebb0 0x7fdf400d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1084023841 0 0) 0x7fdf400d3230 con 0x7fdf400ba560 2024-04-17T06:09:44.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf28003520 con 0x7fdf400ba560 2024-04-17T06:09:44.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 978215978 0 0) 0x7fdf28003520 con 0x7fdf400ba560 2024-04-17T06:09:44.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 >> v2:172.21.15.190:3300/0 conn(0x7fdf4015ebb0 msgr2=0x7fdf400d42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:44.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 --2- 172.21.15.135:0/1693451757 >> v2:172.21.15.190:3300/0 conn(0x7fdf4015ebb0 0x7fdf400d42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 >> v1:172.21.15.190:6789/0 conn(0x7fdf400d1850 legacy=0x7fdf400b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf400d3ad0 con 0x7fdf400ba560 2024-04-17T06:09:44.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fdf34001670 con 0x7fdf400ba560 2024-04-17T06:09:44.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 >> v1:172.21.15.135:6789/0 conn(0x7fdf400ba560 legacy=0x7fdf400d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf280039b0 0x7fdf28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf28004800 0x7fdf28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280051c0 0x7fdf2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf400d37f0 con 0x7fdf280039b0 2024-04-17T06:09:44.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2800c060 con 0x7fdf280051c0 2024-04-17T06:09:44.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2800c260 con 0x7fdf28004800 2024-04-17T06:09:44.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.160+0000 7fdf3f7fe640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf280039b0 0x7fdf28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3effd640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf28004800 0x7fdf28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3ffff640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280051c0 0x7fdf2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3f7fe640 1 -- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf28004800 msgr2=0x7fdf28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:44.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3f7fe640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf28004800 0x7fdf28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.172 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3f7fe640 1 -- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280051c0 msgr2=0x7fdf2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:44.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3f7fe640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280051c0 0x7fdf2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3f7fe640 1 -- 172.21.15.135:0/1693451757 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf28003520 con 0x7fdf280039b0 2024-04-17T06:09:44.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3ffff640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280051c0 0x7fdf2800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:44.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3ffff640 1 -- 172.21.15.135:0/1693451757 reap_dead start 2024-04-17T06:09:44.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.161+0000 7fdf3f7fe640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf280039b0 0x7fdf28003dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009ac0 tx=0x7fdf28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3c362d65235b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30039070 con 0x7fdf280039b0 2024-04-17T06:09:44.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3002c6b0 con 0x7fdf280039b0 2024-04-17T06:09:44.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf3e7fc640 1 -- 172.21.15.135:0/1693451757 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30009690 con 0x7fdf280039b0 2024-04-17T06:09:44.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf45eba640 1 -- 172.21.15.135:0/1693451757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf280039b0 msgr2=0x7fdf28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf45eba640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf280039b0 0x7fdf28003dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009ac0 tx=0x7fdf28005830 comp rx=0 tx=0).stop 2024-04-17T06:09:44.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf45eba640 1 -- 172.21.15.135:0/1693451757 shutdown_connections 2024-04-17T06:09:44.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf45eba640 1 --2- 172.21.15.135:0/1693451757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf280039b0 0x7fdf28003dd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf45eba640 1 -- 172.21.15.135:0/1693451757 >> 172.21.15.135:0/1693451757 conn(0x7fdf4006d270 msgr2=0x7fdf4006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:44.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf45eba640 1 -- 172.21.15.135:0/1693451757 shutdown_connections 2024-04-17T06:09:44.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.162+0000 7fdf45eba640 1 -- 172.21.15.135:0/1693451757 wait complete. 2024-04-17T06:09:44.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.163+0000 7fdf45eba640 1 Processor -- start 2024-04-17T06:09:44.175 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.163+0000 7fdf45eba640 1 -- start start 2024-04-17T06:09:44.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf45eba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280039b0 0x7fdf400ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf45eba640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf400cd340 0x7fdf400cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf45eba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf40153850 0x7fdf40153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf45eba640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf400d4450 con 0x7fdf400cd340 2024-04-17T06:09:44.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf45eba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdf400d42d0 con 0x7fdf280039b0 2024-04-17T06:09:44.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf45eba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdf400d45d0 con 0x7fdf40153850 2024-04-17T06:09:44.176 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf3effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf400cd340 0x7fdf400cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf3effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf400cd340 0x7fdf400cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49142/0 (socket says 172.21.15.135:49142) 2024-04-17T06:09:44.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.164+0000 7fdf3effd640 1 -- 172.21.15.135:0/2583238465 learned_addr learned my addr 172.21.15.135:0/2583238465 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:44.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf3effd640 1 -- 172.21.15.135:0/2583238465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf40153850 msgr2=0x7fdf40153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf3ffff640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf40153850 0x7fdf40153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf3effd640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf40153850 0x7fdf40153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.177 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf3effd640 1 -- 172.21.15.135:0/2583238465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280039b0 msgr2=0x7fdf400ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf3effd640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280039b0 0x7fdf400ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf3effd640 1 -- 172.21.15.135:0/2583238465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf400cde90 con 0x7fdf400cd340 2024-04-17T06:09:44.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf3effd640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf400cd340 0x7fdf400cd7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf34007310 tx=0x7fdf340077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf27fff640 1 -- 172.21.15.135:0/2583238465 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400fa00 con 0x7fdf400cd340 2024-04-17T06:09:44.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf27fff640 1 -- 172.21.15.135:0/2583238465 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf34018070 con 0x7fdf400cd340 2024-04-17T06:09:44.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf27fff640 1 -- 172.21.15.135:0/2583238465 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400fe00 con 0x7fdf400cd340 2024-04-17T06:09:44.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf400ce170 con 0x7fdf400cd340 2024-04-17T06:09:44.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.165+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf40154570 con 0x7fdf400cd340 2024-04-17T06:09:44.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.166+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf40074940 con 0x7fdf400cd340 2024-04-17T06:09:44.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.166+0000 7fdf27fff640 1 -- 172.21.15.135:0/2583238465 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf3401d460 con 0x7fdf400cd340 2024-04-17T06:09:44.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.167+0000 7fdf27fff640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdf2c06c920 0x7fdf2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.167+0000 7fdf27fff640 1 -- 172.21.15.135:0/2583238465 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdf3408c010 con 0x7fdf400cd340 2024-04-17T06:09:44.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.167+0000 7fdf3f7fe640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdf2c06c920 0x7fdf2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.167+0000 7fdf3f7fe640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdf2c06c920 0x7fdf2c06ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf30036000 tx=0x7fdf3002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.171+0000 7fdf27fff640 1 -- 172.21.15.135:0/2583238465 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf340112b0 con 0x7fdf400cd340 2024-04-17T06:09:44.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.344+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdf400cf050 con 0x7fdf400cd340 2024-04-17T06:09:44.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.345+0000 7fdf27fff640 1 -- 172.21.15.135:0/2583238465 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7fdf3405a100 con 0x7fdf400cd340 2024-04-17T06:09:44.343 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:44.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdf2c06c920 msgr2=0x7fdf2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdf2c06c920 0x7fdf2c06ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf30036000 tx=0x7fdf3002e000 comp rx=0 tx=0).stop 2024-04-17T06:09:44.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf400cd340 msgr2=0x7fdf400cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf400cd340 0x7fdf400cd7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf34007310 tx=0x7fdf340077d0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 shutdown_connections 2024-04-17T06:09:44.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf40153850 0x7fdf40153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdf2c06c920 0x7fdf2c06ede0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf400cd340 0x7fdf400cd7c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 --2- 172.21.15.135:0/2583238465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf280039b0 0x7fdf400ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 >> 172.21.15.135:0/2583238465 conn(0x7fdf4006d270 msgr2=0x7fdf400b8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:44.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 shutdown_connections 2024-04-17T06:09:44.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.347+0000 7fdf45eba640 1 -- 172.21.15.135:0/2583238465 wait complete. 2024-04-17T06:09:44.356 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:09:44.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.461+0000 7fd149460640 1 Processor -- start 2024-04-17T06:09:44.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.461+0000 7fd149460640 1 -- start start 2024-04-17T06:09:44.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.462+0000 7fd149460640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd1440ba560 0x7fd1440d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.462+0000 7fd149460640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd1440d1850 0x7fd1440d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.462+0000 7fd149460640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1440d3230 con 0x7fd14415ebb0 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.462+0000 7fd149460640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd1440d21e0 con 0x7fd1440ba560 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.462+0000 7fd149460640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd1440d3510 con 0x7fd1440d1850 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.462+0000 7fd142ffd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd14415ebb0 0x7fd1440d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:37876/0 (socket says 172.21.15.135:37876) 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.462+0000 7fd142ffd640 1 -- 172.21.15.135:0/543364809 learned_addr learned my addr 172.21.15.135:0/543364809 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.462+0000 7fd13a7fc640 1 --2- 172.21.15.135:0/543364809 >> v2:172.21.15.135:3300/0 conn(0x7fd1440ba560 0x7fd1440d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.463+0000 7fd1437fe640 1 --2- 172.21.15.135:0/543364809 >> v2:172.21.15.190:3300/0 conn(0x7fd1440d1850 0x7fd1440d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.463+0000 7fd13a7fc640 1 -- 172.21.15.135:0/543364809 >> v2:172.21.15.190:3300/0 conn(0x7fd1440d1850 msgr2=0x7fd1440d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.463+0000 7fd13a7fc640 1 --2- 172.21.15.135:0/543364809 >> v2:172.21.15.190:3300/0 conn(0x7fd1440d1850 0x7fd1440d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.463+0000 7fd13a7fc640 1 -- 172.21.15.135:0/543364809 >> v1:172.21.15.135:6789/0 conn(0x7fd14415ebb0 legacy=0x7fd1440d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.463+0000 7fd13a7fc640 1 -- 172.21.15.135:0/543364809 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1440d37f0 con 0x7fd1440ba560 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.463+0000 7fd1427fc640 1 -- 172.21.15.135:0/543364809 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370375698 0 0) 0x7fd1440d3230 con 0x7fd14415ebb0 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.463+0000 7fd13a7fc640 1 --2- 172.21.15.135:0/543364809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1440ba560 0x7fd1440d42d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1440ceef0 tx=0x7fd12c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bcf48ef1c54930 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd1427fc640 1 -- 172.21.15.135:0/543364809 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c035020 con 0x7fd1440ba560 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd1427fc640 1 -- 172.21.15.135:0/543364809 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c031830 con 0x7fd1440ba560 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd1427fc640 1 -- 172.21.15.135:0/543364809 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c031c00 con 0x7fd1440ba560 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd149460640 1 -- 172.21.15.135:0/543364809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1440ba560 msgr2=0x7fd1440d42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd149460640 1 --2- 172.21.15.135:0/543364809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1440ba560 0x7fd1440d42d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1440ceef0 tx=0x7fd12c02b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd149460640 1 -- 172.21.15.135:0/543364809 shutdown_connections 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd149460640 1 --2- 172.21.15.135:0/543364809 >> v2:172.21.15.190:3300/0 conn(0x7fd1440d1850 0x7fd1440d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd149460640 1 --2- 172.21.15.135:0/543364809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd1440ba560 0x7fd1440d42d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd149460640 1 -- 172.21.15.135:0/543364809 >> 172.21.15.135:0/543364809 conn(0x7fd14406d270 msgr2=0x7fd14406d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:44.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd149460640 1 -- 172.21.15.135:0/543364809 shutdown_connections 2024-04-17T06:09:44.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.464+0000 7fd149460640 1 -- 172.21.15.135:0/543364809 wait complete. 2024-04-17T06:09:44.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.465+0000 7fd149460640 1 Processor -- start 2024-04-17T06:09:44.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.465+0000 7fd149460640 1 -- start start 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd149460640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1440ba560 0x7fd1440cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd149460640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440d1850 0x7fd1441f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd149460640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415ebb0 0x7fd1440cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd149460640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd1440b7a40 con 0x7fd14415ebb0 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd149460640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd1440d3db0 con 0x7fd1440ba560 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd149460640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd1440b7bc0 con 0x7fd1440d1850 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415ebb0 0x7fd1440cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415ebb0 0x7fd1440cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49164/0 (socket says 172.21.15.135:49164) 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.466+0000 7fd1437fe640 1 -- 172.21.15.135:0/4016328310 learned_addr learned my addr 172.21.15.135:0/4016328310 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd13a7fc640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440d1850 0x7fd1441f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd142ffd640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1440ba560 0x7fd1440cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd1437fe640 1 -- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440d1850 msgr2=0x7fd1441f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd1437fe640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440d1850 0x7fd1441f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd1437fe640 1 -- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1440ba560 msgr2=0x7fd1440cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd1437fe640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1440ba560 0x7fd1440cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd1437fe640 1 -- 172.21.15.135:0/4016328310 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1440cc230 con 0x7fd14415ebb0 2024-04-17T06:09:44.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd142ffd640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1440ba560 0x7fd1440cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:44.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd1437fe640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415ebb0 0x7fd1440cb9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00cda0 tx=0x7fd13c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd13bfff640 1 -- 172.21.15.135:0/4016328310 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c003b90 con 0x7fd14415ebb0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd13bfff640 1 -- 172.21.15.135:0/4016328310 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd13c003d30 con 0x7fd14415ebb0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1440cc510 con 0x7fd14415ebb0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd13bfff640 1 -- 172.21.15.135:0/4016328310 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c004240 con 0x7fd14415ebb0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.467+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd14415f130 con 0x7fd14415ebb0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.468+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd144074940 con 0x7fd14415ebb0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.469+0000 7fd13bfff640 1 -- 172.21.15.135:0/4016328310 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd13c0043e0 con 0x7fd14415ebb0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.469+0000 7fd13bfff640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12406c8f0 0x7fd12406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.469+0000 7fd142ffd640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12406c8f0 0x7fd12406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.469+0000 7fd13bfff640 1 -- 172.21.15.135:0/4016328310 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd13c08c950 con 0x7fd14415ebb0 2024-04-17T06:09:44.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.470+0000 7fd142ffd640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12406c8f0 0x7fd12406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd134000f60 tx=0x7fd134009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.474+0000 7fd13bfff640 1 -- 172.21.15.135:0/4016328310 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd13c05aa40 con 0x7fd14415ebb0 2024-04-17T06:09:44.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.687+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1441f4890 con 0x7fd14415ebb0 2024-04-17T06:09:44.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.688+0000 7fd13bfff640 1 -- 172.21.15.135:0/4016328310 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fd13c05a340 con 0x7fd14415ebb0 2024-04-17T06:09:44.686 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:44.686 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:36.038772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:09:44.687 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 9 2024-04-17T06:09:44.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12406c8f0 msgr2=0x7fd12406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12406c8f0 0x7fd12406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd134000f60 tx=0x7fd134009040 comp rx=0 tx=0).stop 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415ebb0 msgr2=0x7fd1440cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415ebb0 0x7fd1440cb9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00cda0 tx=0x7fd13c00a660 comp rx=0 tx=0).stop 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 shutdown_connections 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd12406c8f0 0x7fd12406edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd14415ebb0 0x7fd1440cb9e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1440d1850 0x7fd1441f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 --2- 172.21.15.135:0/4016328310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1440ba560 0x7fd1440cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 >> 172.21.15.135:0/4016328310 conn(0x7fd14406d270 msgr2=0x7fd1440d48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 shutdown_connections 2024-04-17T06:09:44.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.690+0000 7fd149460640 1 -- 172.21.15.135:0/4016328310 wait complete. 2024-04-17T06:09:44.700 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:44.803 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.805+0000 7f16950d9640 1 Processor -- start 2024-04-17T06:09:44.803 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.805+0000 7f16950d9640 1 -- start start 2024-04-17T06:09:44.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.805+0000 7f16950d9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f16900725d0 0x7f169015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.805+0000 7f16950d9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16900cf740 0x7f16900cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.805+0000 7f16950d9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f169006e170 con 0x7f169015ff30 2024-04-17T06:09:44.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.805+0000 7f16950d9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f169006df20 con 0x7f16900725d0 2024-04-17T06:09:44.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.805+0000 7f16950d9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f169006eaa0 con 0x7f16900cf740 2024-04-17T06:09:44.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168ed9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f169015ff30 0x7f1690071ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38638/0 (socket says 172.21.15.135:38638) 2024-04-17T06:09:44.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168ed9d640 1 -- 172.21.15.135:0/4095738034 learned_addr learned my addr 172.21.15.135:0/4095738034 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:44.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168e59c640 1 --2- 172.21.15.135:0/4095738034 >> v2:172.21.15.190:3300/0 conn(0x7f16900725d0 0x7f169015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168f59e640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16900cf740 0x7f16900cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399695239 0 0) 0x7f169006e170 con 0x7f169015ff30 2024-04-17T06:09:44.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f167c003520 con 0x7f169015ff30 2024-04-17T06:09:44.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1963165887 0 0) 0x7f167c003520 con 0x7f169015ff30 2024-04-17T06:09:44.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16900cf740 msgr2=0x7f16900cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16900cf740 0x7f16900cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 >> v2:172.21.15.190:3300/0 conn(0x7f16900725d0 msgr2=0x7f169015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 --2- 172.21.15.135:0/4095738034 >> v2:172.21.15.190:3300/0 conn(0x7f16900725d0 0x7f169015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169006e450 con 0x7f169015ff30 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f1680001670 con 0x7f169015ff30 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 >> v1:172.21.15.135:6789/0 conn(0x7f169015ff30 legacy=0x7f1690071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16900cf740 0x7f167c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c003f30 0x7f167c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f167c004d40 0x7f167c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f169006df20 con 0x7f167c004d40 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f169006eaa0 con 0x7f167c003f30 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f167c00b9c0 con 0x7f16900cf740 2024-04-17T06:09:44.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168f59e640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f167c004d40 0x7f167c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168ed9d640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16900cf740 0x7f167c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.806+0000 7f168e59c640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c003f30 0x7f167c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168f59e640 1 -- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16900cf740 msgr2=0x7f167c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168f59e640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f16900cf740 0x7f167c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168f59e640 1 -- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c003f30 msgr2=0x7f167c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168f59e640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f167c003f30 0x7f167c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168f59e640 1 -- 172.21.15.135:0/4095738034 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c003520 con 0x7f167c004d40 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168f59e640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f167c004d40 0x7f167c0051c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1688009fc0 tx=0x7f167c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d8b38d60870e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688039070 con 0x7f167c004d40 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168802cc00 con 0x7f167c004d40 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.807+0000 7f168dd9b640 1 -- 172.21.15.135:0/4095738034 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688009c10 con 0x7f167c004d40 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.808+0000 7f16950d9640 1 -- 172.21.15.135:0/4095738034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f167c004d40 msgr2=0x7f167c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.808+0000 7f16950d9640 1 --2- 172.21.15.135:0/4095738034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f167c004d40 0x7f167c0051c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1688009fc0 tx=0x7f167c005830 comp rx=0 tx=0).stop 2024-04-17T06:09:44.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.808+0000 7f168f59e640 1 -- 172.21.15.135:0/4095738034 reap_dead start 2024-04-17T06:09:44.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.808+0000 7f16950d9640 1 -- 172.21.15.135:0/4095738034 shutdown_connections 2024-04-17T06:09:44.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.808+0000 7f16950d9640 1 -- 172.21.15.135:0/4095738034 >> 172.21.15.135:0/4095738034 conn(0x7f16900d0520 msgr2=0x7f16900d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:44.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.808+0000 7f16950d9640 1 -- 172.21.15.135:0/4095738034 shutdown_connections 2024-04-17T06:09:44.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.808+0000 7f16950d9640 1 -- 172.21.15.135:0/4095738034 wait complete. 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.809+0000 7f16950d9640 1 Processor -- start 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.810+0000 7f16950d9640 1 -- start start 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.810+0000 7f16950d9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f167c004d40 0x7f169015bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.810+0000 7f16950d9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f169013d840 0x7f16900c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.810+0000 7f16950d9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f169013dcc0 0x7f16900c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.810+0000 7f16950d9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f16900cfd70 con 0x7f169013dcc0 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.810+0000 7f16950d9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f16900cfbf0 con 0x7f169013d840 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.810+0000 7f16950d9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f16900cfef0 con 0x7f167c004d40 2024-04-17T06:09:44.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f169013dcc0 0x7f16900c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168ed9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f167c004d40 0x7f169015bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168e59c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f169013d840 0x7f16900c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f169013dcc0 0x7f16900c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37620/0 (socket says 172.21.15.135:37620) 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 -- 172.21.15.135:0/1588618399 learned_addr learned my addr 172.21.15.135:0/1588618399 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 -- 172.21.15.135:0/1588618399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f167c004d40 msgr2=0x7f169015bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f167c004d40 0x7f169015bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 -- 172.21.15.135:0/1588618399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f169013d840 msgr2=0x7f16900c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f169013d840 0x7f16900c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 -- 172.21.15.135:0/1588618399 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169013fce0 con 0x7f169013dcc0 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f168f59e640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f169013dcc0 0x7f16900c5c00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1688037040 tx=0x7f1688009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f16877fe640 1 -- 172.21.15.135:0/1588618399 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688039030 con 0x7f169013dcc0 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f16877fe640 1 -- 172.21.15.135:0/1588618399 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168803a040 con 0x7f169013dcc0 2024-04-17T06:09:44.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169013e260 con 0x7f169013dcc0 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f16877fe640 1 -- 172.21.15.135:0/1588618399 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688009c10 con 0x7f169013dcc0 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.811+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169013ff80 con 0x7f169013dcc0 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.812+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1690140180 con 0x7f169013dcc0 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.813+0000 7f16877fe640 1 -- 172.21.15.135:0/1588618399 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1688045030 con 0x7f169013dcc0 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.813+0000 7f16877fe640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f167006c920 0x7f167006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.813+0000 7f168ed9d640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f167006c920 0x7f167006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.813+0000 7f16877fe640 1 -- 172.21.15.135:0/1588618399 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f16880ada50 con 0x7f169013dcc0 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.814+0000 7f168ed9d640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f167006c920 0x7f167006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1680007dd0 tx=0x7f16800078f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:44.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.818+0000 7f16877fe640 1 -- 172.21.15.135:0/1588618399 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1690140180 con 0x7f169013dcc0 2024-04-17T06:09:44.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.992+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1690074940 con 0x7f169013dcc0 2024-04-17T06:09:44.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.993+0000 7f16877fe640 1 -- 172.21.15.135:0/1588618399 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f168804c060 con 0x7f169013dcc0 2024-04-17T06:09:44.991 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:44.992 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f167006c920 msgr2=0x7f167006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f167006c920 0x7f167006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1680007dd0 tx=0x7f16800078f0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f169013dcc0 msgr2=0x7f16900c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f169013dcc0 0x7f16900c5c00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1688037040 tx=0x7f1688009750 comp rx=0 tx=0).stop 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 shutdown_connections 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f167006c920 0x7f167006ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f169013dcc0 0x7f16900c5c00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f169013d840 0x7f16900c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 --2- 172.21.15.135:0/1588618399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f167c004d40 0x7f169015bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 >> 172.21.15.135:0/1588618399 conn(0x7f16900d0520 msgr2=0x7f16900cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:44.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.995+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 shutdown_connections 2024-04-17T06:09:44.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:44.996+0000 7f16950d9640 1 -- 172.21.15.135:0/1588618399 wait complete. 2024-04-17T06:09:45.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:09:45.005 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:45.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.114+0000 7fd4f4de9640 1 Processor -- start 2024-04-17T06:09:45.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.114+0000 7fd4f4de9640 1 -- start start 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4f4de9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4f0165360 0x7fd4f0167750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4f4de9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4f006ba30 con 0x7fd4f00cd0a0 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4f4de9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4f006bcd0 con 0x7fd4f015c8a0 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4f4de9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f006c600 con 0x7fd4f0165360 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4e5d9b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd4f00cd0a0 0x7fd4f00cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38646/0 (socket says 172.21.15.135:38646) 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4e5d9b640 1 -- 172.21.15.135:0/2017593248 learned_addr learned my addr 172.21.15.135:0/2017593248 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4eed9d640 1 --2- 172.21.15.135:0/2017593248 >> v2:172.21.15.190:3300/0 conn(0x7fd4f0165360 0x7fd4f0167750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3595457161 0 0) 0x7fd4f006ba30 con 0x7fd4f00cd0a0 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4dc003520 con 0x7fd4f00cd0a0 2024-04-17T06:09:45.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 488498250 0 0) 0x7fd4dc003520 con 0x7fd4f00cd0a0 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 >> v2:172.21.15.190:3300/0 conn(0x7fd4f0165360 msgr2=0x7fd4f0167750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 --2- 172.21.15.135:0/2017593248 >> v2:172.21.15.190:3300/0 conn(0x7fd4f0165360 0x7fd4f0167750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 >> v1:172.21.15.190:6789/0 conn(0x7fd4f015c8a0 legacy=0x7fd4f00b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f006bfb0 con 0x7fd4f00cd0a0 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fd4d8001670 con 0x7fd4f00cd0a0 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 >> v1:172.21.15.135:6789/0 conn(0x7fd4f00cd0a0 legacy=0x7fd4f00cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.115+0000 7fd4edd9b640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0039b0 0x7fd4dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4edd9b640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc004800 0x7fd4dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4edd9b640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc0051c0 0x7fd4dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f006c600 con 0x7fd4dc0051c0 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc00c160 con 0x7fd4dc004800 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4dc00c360 con 0x7fd4dc0039b0 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4eed9d640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc0051c0 0x7fd4dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4e5d9b640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc004800 0x7fd4dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4ee59c640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0039b0 0x7fd4dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4eed9d640 1 -- 172.21.15.135:0/2017593248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0039b0 msgr2=0x7fd4dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4eed9d640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0039b0 0x7fd4dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4eed9d640 1 -- 172.21.15.135:0/2017593248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc004800 msgr2=0x7fd4dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4eed9d640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4dc004800 0x7fd4dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.116+0000 7fd4eed9d640 1 -- 172.21.15.135:0/2017593248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc003520 con 0x7fd4dc0051c0 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4eed9d640 1 -- 172.21.15.135:0/2017593248 reap_dead start 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4eed9d640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc0051c0 0x7fd4dc00b9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00d1460 tx=0x7fd4dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e55752d9faada7a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8009c10 con 0x7fd4dc0051c0 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e802cbd0 con 0x7fd4dc0051c0 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4edd9b640 1 -- 172.21.15.135:0/2017593248 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8003990 con 0x7fd4dc0051c0 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4f4de9640 1 -- 172.21.15.135:0/2017593248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc0051c0 msgr2=0x7fd4dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4f4de9640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc0051c0 0x7fd4dc00b9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00d1460 tx=0x7fd4dc005830 comp rx=0 tx=0).stop 2024-04-17T06:09:45.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4f4de9640 1 -- 172.21.15.135:0/2017593248 shutdown_connections 2024-04-17T06:09:45.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4f4de9640 1 --2- 172.21.15.135:0/2017593248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4dc0051c0 0x7fd4dc00b9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4f4de9640 1 -- 172.21.15.135:0/2017593248 >> 172.21.15.135:0/2017593248 conn(0x7fd4f009a9f0 msgr2=0x7fd4f00b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:45.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4f4de9640 1 -- 172.21.15.135:0/2017593248 shutdown_connections 2024-04-17T06:09:45.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.117+0000 7fd4f4de9640 1 -- 172.21.15.135:0/2017593248 wait complete. 2024-04-17T06:09:45.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.119+0000 7fd4f4de9640 1 Processor -- start 2024-04-17T06:09:45.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.119+0000 7fd4f4de9640 1 -- start start 2024-04-17T06:09:45.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4f4de9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0051c0 0x7fd4f0157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4f4de9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f00c6230 0x7fd4f00af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4f4de9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f00af960 0x7fd4f00c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4f4de9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f00d0370 con 0x7fd4f00af960 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4f4de9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f00ced60 con 0x7fd4f00c6230 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4f4de9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f00d04f0 con 0x7fd4dc0051c0 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f00af960 0x7fd4f00c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f00af960 0x7fd4f00c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37648/0 (socket says 172.21.15.135:37648) 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0051c0 0x7fd4f0157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 -- 172.21.15.135:0/1888676963 learned_addr learned my addr 172.21.15.135:0/1888676963 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4e5d9b640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f00c6230 0x7fd4f00af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 -- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0051c0 msgr2=0x7fd4f0157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0051c0 0x7fd4f0157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 -- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f00c6230 msgr2=0x7fd4f00af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f00c6230 0x7fd4f00af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 -- 172.21.15.135:0/1888676963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f015dd30 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4ee59c640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0051c0 0x7fd4f0157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4e5d9b640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f00c6230 0x7fd4f00af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4eed9d640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f00af960 0x7fd4f00c73c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4e802c640 tx=0x7fd4e8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4e77fe640 1 -- 172.21.15.135:0/1888676963 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8003990 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4e77fe640 1 -- 172.21.15.135:0/1888676963 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e8044070 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.120+0000 7fd4e77fe640 1 -- 172.21.15.135:0/1888676963 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8003d50 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.121+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f015e010 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.121+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f00c6ea0 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.121+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f00c70a0 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.122+0000 7fd4e77fe640 1 -- 172.21.15.135:0/1888676963 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4e8005d10 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.122+0000 7fd4e77fe640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.122+0000 7fd4e77fe640 1 -- 172.21.15.135:0/1888676963 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd4e80ae3e0 con 0x7fd4f00af960 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.123+0000 7fd4ee59c640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.123+0000 7fd4ee59c640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4d006c8f0 0x7fd4d006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0000c00 tx=0x7fd4e0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.127+0000 7fd4e77fe640 1 -- 172.21.15.135:0/1888676963 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4f00c70a0 con 0x7fd4f00af960 2024-04-17T06:09:45.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.301+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4f0074940 con 0x7fd4f00af960 2024-04-17T06:09:45.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.302+0000 7fd4e77fe640 1 -- 172.21.15.135:0/1888676963 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fd4e807bf10 con 0x7fd4f00af960 2024-04-17T06:09:45.300 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:45.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4d006c8f0 msgr2=0x7fd4d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4d006c8f0 0x7fd4d006edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0000c00 tx=0x7fd4e0009040 comp rx=0 tx=0).stop 2024-04-17T06:09:45.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f00af960 msgr2=0x7fd4f00c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f00af960 0x7fd4f00c73c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4e802c640 tx=0x7fd4e8009750 comp rx=0 tx=0).stop 2024-04-17T06:09:45.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 shutdown_connections 2024-04-17T06:09:45.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f00af960 0x7fd4f00c73c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4f00c6230 0x7fd4f00af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 --2- 172.21.15.135:0/1888676963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4dc0051c0 0x7fd4f0157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 >> 172.21.15.135:0/1888676963 conn(0x7fd4f009a9f0 msgr2=0x7fd4f00cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:45.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 shutdown_connections 2024-04-17T06:09:45.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.304+0000 7fd4f4de9640 1 -- 172.21.15.135:0/1888676963 wait complete. 2024-04-17T06:09:45.313 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:09:45.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.422+0000 7fca870d9640 1 Processor -- start 2024-04-17T06:09:45.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.422+0000 7fca870d9640 1 -- start start 2024-04-17T06:09:45.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca870d9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fca8006b990 0x7fca800cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca870d9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca800cfc40 0x7fca8015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca870d9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca800b5ae0 con 0x7fca8015ec70 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca870d9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fca8006bdf0 con 0x7fca8006b990 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca870d9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca800b60a0 con 0x7fca800cfc40 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca7ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fca8006b990 0x7fca800cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca7ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fca8006b990 0x7fca800cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37656/0 (socket says 172.21.15.135:37656) 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca7ffff640 1 -- 172.21.15.135:0/463526556 learned_addr learned my addr 172.21.15.135:0/463526556 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.423+0000 7fca85676640 1 --2- 172.21.15.135:0/463526556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca800cfc40 0x7fca8015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.424+0000 7fca7ffff640 1 -- 172.21.15.135:0/463526556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca800cfc40 msgr2=0x7fca8015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.424+0000 7fca7ffff640 1 --2- 172.21.15.135:0/463526556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca800cfc40 0x7fca8015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.424+0000 7fca7ffff640 1 -- 172.21.15.135:0/463526556 >> v1:172.21.15.190:6789/0 conn(0x7fca8015ec70 legacy=0x7fca800d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.424+0000 7fca7ffff640 1 -- 172.21.15.135:0/463526556 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca800b5dc0 con 0x7fca8006b990 2024-04-17T06:09:45.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.424+0000 7fca85676640 1 --2- 172.21.15.135:0/463526556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca800cfc40 0x7fca8015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.424+0000 7fca7ffff640 1 --2- 172.21.15.135:0/463526556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca800cf700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca74009870 tx=0x7fca7402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e94dc2703b0ffe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.424+0000 7fca7f7fe640 1 -- 172.21.15.135:0/463526556 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca740317c0 con 0x7fca8006b990 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca7f7fe640 1 -- 172.21.15.135:0/463526556 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca74037070 con 0x7fca8006b990 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca7f7fe640 1 -- 172.21.15.135:0/463526556 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74031b80 con 0x7fca8006b990 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca870d9640 1 -- 172.21.15.135:0/463526556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 msgr2=0x7fca800cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca870d9640 1 --2- 172.21.15.135:0/463526556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca800cf700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca74009870 tx=0x7fca7402b860 comp rx=0 tx=0).stop 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca870d9640 1 -- 172.21.15.135:0/463526556 shutdown_connections 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca870d9640 1 --2- 172.21.15.135:0/463526556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca800cfc40 0x7fca8015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca870d9640 1 --2- 172.21.15.135:0/463526556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca800cf700 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca870d9640 1 -- 172.21.15.135:0/463526556 >> 172.21.15.135:0/463526556 conn(0x7fca800cd7e0 msgr2=0x7fca800cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:45.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca870d9640 1 -- 172.21.15.135:0/463526556 shutdown_connections 2024-04-17T06:09:45.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.425+0000 7fca870d9640 1 -- 172.21.15.135:0/463526556 wait complete. 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.426+0000 7fca870d9640 1 Processor -- start 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.426+0000 7fca870d9640 1 -- start start 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca870d9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca80157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca870d9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca800cfc40 0x7fca8015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca870d9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca8015ec70 0x7fca8015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca870d9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca800d00b0 con 0x7fca8006b990 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca870d9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca800b4160 con 0x7fca800cfc40 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca870d9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca800d0230 con 0x7fca8015ec70 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca84e75640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca80157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca84e75640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca80157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37660/0 (socket says 172.21.15.135:37660) 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca84e75640 1 -- 172.21.15.135:0/769345207 learned_addr learned my addr 172.21.15.135:0/769345207 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:45.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca85676640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca8015ec70 0x7fca8015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.427+0000 7fca7ffff640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca800cfc40 0x7fca8015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca84e75640 1 -- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca8015ec70 msgr2=0x7fca8015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca84e75640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca8015ec70 0x7fca8015c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca84e75640 1 -- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca800cfc40 msgr2=0x7fca8015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca84e75640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca800cfc40 0x7fca8015c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca84e75640 1 -- 172.21.15.135:0/769345207 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca80158660 con 0x7fca8006b990 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca7ffff640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca800cfc40 0x7fca8015c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca84e75640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca80157bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca7000c460 tx=0x7fca7000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca7d7fa640 1 -- 172.21.15.135:0/769345207 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70016020 con 0x7fca8006b990 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca7d7fa640 1 -- 172.21.15.135:0/769345207 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca70012870 con 0x7fca8006b990 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca7d7fa640 1 -- 172.21.15.135:0/769345207 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70012c60 con 0x7fca8006b990 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca80158940 con 0x7fca8006b990 2024-04-17T06:09:45.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.428+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca80158f00 con 0x7fca8006b990 2024-04-17T06:09:45.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.429+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca80074940 con 0x7fca8006b990 2024-04-17T06:09:45.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.429+0000 7fca7d7fa640 1 -- 172.21.15.135:0/769345207 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca70019020 con 0x7fca8006b990 2024-04-17T06:09:45.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.430+0000 7fca7d7fa640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca6c06c8f0 0x7fca6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.430+0000 7fca7d7fa640 1 -- 172.21.15.135:0/769345207 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fca7008ba90 con 0x7fca8006b990 2024-04-17T06:09:45.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.430+0000 7fca7ffff640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca6c06c8f0 0x7fca6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.430+0000 7fca7ffff640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca6c06c8f0 0x7fca6c06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca74036000 tx=0x7fca7403d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.434+0000 7fca7d7fa640 1 -- 172.21.15.135:0/769345207 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca70020030 con 0x7fca8006b990 2024-04-17T06:09:45.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.607+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fca800ce810 con 0x7fca8006b990 2024-04-17T06:09:45.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.607+0000 7fca7d7fa640 1 -- 172.21.15.135:0/769345207 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7fca70059c10 con 0x7fca8006b990 2024-04-17T06:09:45.605 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:45.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.609+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca6c06c8f0 msgr2=0x7fca6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.609+0000 7fca870d9640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca6c06c8f0 0x7fca6c06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca74036000 tx=0x7fca7403d040 comp rx=0 tx=0).stop 2024-04-17T06:09:45.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.609+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 msgr2=0x7fca80157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.609+0000 7fca870d9640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca80157bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fca7000c460 tx=0x7fca7000c920 comp rx=0 tx=0).stop 2024-04-17T06:09:45.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.610+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 shutdown_connections 2024-04-17T06:09:45.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.610+0000 7fca870d9640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca8015ec70 0x7fca8015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.610+0000 7fca870d9640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca800cfc40 0x7fca8015c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.610+0000 7fca870d9640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca6c06c8f0 0x7fca6c06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.610+0000 7fca870d9640 1 --2- 172.21.15.135:0/769345207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca8006b990 0x7fca80157bf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.610+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 >> 172.21.15.135:0/769345207 conn(0x7fca800cd7e0 msgr2=0x7fca800cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:45.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.610+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 shutdown_connections 2024-04-17T06:09:45.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.610+0000 7fca870d9640 1 -- 172.21.15.135:0/769345207 wait complete. 2024-04-17T06:09:45.619 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:09:45.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.728+0000 7fed967f4640 1 Processor -- start 2024-04-17T06:09:45.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.728+0000 7fed967f4640 1 -- start start 2024-04-17T06:09:45.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.728+0000 7fed967f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed900cebf0 0x7fed90073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.728+0000 7fed967f4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fed90073580 0x7fed900b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.728+0000 7fed967f4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed900b75c0 con 0x7fed90144f70 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.728+0000 7fed967f4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fed900739d0 con 0x7fed90073580 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.728+0000 7fed967f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed90160070 con 0x7fed900cebf0 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8ffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fed90144f70 0x7fed9015f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38658/0 (socket says 172.21.15.135:38658) 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8ffff640 1 -- 172.21.15.135:0/3202567287 learned_addr learned my addr 172.21.15.135:0/3202567287 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed87fff640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed900cebf0 0x7fed90073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed94d91640 1 --2- 172.21.15.135:0/3202567287 >> v2:172.21.15.190:3300/0 conn(0x7fed90073580 0x7fed900b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3985965474 0 0) 0x7fed900b75c0 con 0x7fed90144f70 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed7c003520 con 0x7fed90144f70 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3956743934 0 0) 0x7fed7c003520 con 0x7fed90144f70 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed900cebf0 msgr2=0x7fed90073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed900cebf0 0x7fed90073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 >> v2:172.21.15.190:3300/0 conn(0x7fed90073580 msgr2=0x7fed900b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 --2- 172.21.15.135:0/3202567287 >> v2:172.21.15.190:3300/0 conn(0x7fed90073580 0x7fed900b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed90160350 con 0x7fed90144f70 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fed80001670 con 0x7fed90144f70 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 >> v1:172.21.15.135:6789/0 conn(0x7fed90144f70 legacy=0x7fed9015f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0039b0 0x7fed7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed7c004800 0x7fed7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed900cebf0 0x7fed7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed900739d0 con 0x7fed7c004800 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed90160070 con 0x7fed7c0039b0 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed7c00b9f0 con 0x7fed900cebf0 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed87fff640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed7c004800 0x7fed7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed8ffff640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0039b0 0x7fed7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.729+0000 7fed94d91640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed900cebf0 0x7fed7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.730+0000 7fed87fff640 1 -- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed900cebf0 msgr2=0x7fed7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.730+0000 7fed87fff640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed900cebf0 0x7fed7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.730+0000 7fed87fff640 1 -- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0039b0 msgr2=0x7fed7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.730+0000 7fed87fff640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0039b0 0x7fed7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.730+0000 7fed87fff640 1 -- 172.21.15.135:0/3202567287 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c003520 con 0x7fed7c004800 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.730+0000 7fed87fff640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed7c004800 0x7fed7c004c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed78009ac0 tx=0x7fed7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8273c3eaf31f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78039070 con 0x7fed7c004800 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7802c6b0 con 0x7fed7c004800 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed8f7fe640 1 -- 172.21.15.135:0/3202567287 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78009690 con 0x7fed7c004800 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed967f4640 1 -- 172.21.15.135:0/3202567287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed7c004800 msgr2=0x7fed7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed967f4640 1 --2- 172.21.15.135:0/3202567287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed7c004800 0x7fed7c004c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fed78009ac0 tx=0x7fed7c005830 comp rx=0 tx=0).stop 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed94d91640 1 -- 172.21.15.135:0/3202567287 reap_dead start 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed967f4640 1 -- 172.21.15.135:0/3202567287 shutdown_connections 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed967f4640 1 -- 172.21.15.135:0/3202567287 >> 172.21.15.135:0/3202567287 conn(0x7fed900d15e0 msgr2=0x7fed900d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:45.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed967f4640 1 -- 172.21.15.135:0/3202567287 shutdown_connections 2024-04-17T06:09:45.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.731+0000 7fed967f4640 1 -- 172.21.15.135:0/3202567287 wait complete. 2024-04-17T06:09:45.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.732+0000 7fed967f4640 1 Processor -- start 2024-04-17T06:09:45.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.732+0000 7fed967f4640 1 -- start start 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed967f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c004800 0x7fed900c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed967f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed90142c40 0x7fed90142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed967f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed900b2c20 0x7fed900b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed967f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed900cf050 con 0x7fed90142c40 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed967f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fed900ce3f0 con 0x7fed900b2c20 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed967f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fed9015fc10 con 0x7fed7c004800 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed87fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed90142c40 0x7fed90142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed87fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed90142c40 0x7fed90142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37682/0 (socket says 172.21.15.135:37682) 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed87fff640 1 -- 172.21.15.135:0/279317027 learned_addr learned my addr 172.21.15.135:0/279317027 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed8ffff640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c004800 0x7fed900c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.733+0000 7fed94d91640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed900b2c20 0x7fed900b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed87fff640 1 -- 172.21.15.135:0/279317027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c004800 msgr2=0x7fed900c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed87fff640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c004800 0x7fed900c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed87fff640 1 -- 172.21.15.135:0/279317027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed900b2c20 msgr2=0x7fed900b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed87fff640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed900b2c20 0x7fed900b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed87fff640 1 -- 172.21.15.135:0/279317027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed900c9410 con 0x7fed90142c40 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed87fff640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed90142c40 0x7fed90142520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed78000c00 tx=0x7fed78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed8d7fa640 1 -- 172.21.15.135:0/279317027 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78042020 con 0x7fed90142c40 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed8d7fa640 1 -- 172.21.15.135:0/279317027 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7803a040 con 0x7fed90142c40 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed900c96f0 con 0x7fed90142c40 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed8d7fa640 1 -- 172.21.15.135:0/279317027 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78039070 con 0x7fed90142c40 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.734+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed900c9cb0 con 0x7fed90142c40 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.735+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed90068a20 con 0x7fed90142c40 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.735+0000 7fed8d7fa640 1 -- 172.21.15.135:0/279317027 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed78045020 con 0x7fed90142c40 2024-04-17T06:09:45.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.736+0000 7fed8d7fa640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fed7006c920 0x7fed7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:45.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.736+0000 7fed8ffff640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fed7006c920 0x7fed7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:45.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.736+0000 7fed8d7fa640 1 -- 172.21.15.135:0/279317027 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fed780acb60 con 0x7fed90142c40 2024-04-17T06:09:45.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.737+0000 7fed8ffff640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fed7006c920 0x7fed7006ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed80001050 tx=0x7fed80007970 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:45.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.741+0000 7fed8d7fa640 1 -- 172.21.15.135:0/279317027 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed7807a600 con 0x7fed90142c40 2024-04-17T06:09:45.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.914+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fed900b12a0 con 0x7fed90142c40 2024-04-17T06:09:45.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.914+0000 7fed8d7fa640 1 -- 172.21.15.135:0/279317027 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fed7807e900 con 0x7fed90142c40 2024-04-17T06:09:45.913 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fed7006c920 msgr2=0x7fed7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fed7006c920 0x7fed7006ede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed80001050 tx=0x7fed80007970 comp rx=0 tx=0).stop 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed90142c40 msgr2=0x7fed90142520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed90142c40 0x7fed90142520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed78000c00 tx=0x7fed78004730 comp rx=0 tx=0).stop 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 shutdown_connections 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed900b2c20 0x7fed900b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fed7006c920 0x7fed7006ede0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed90142c40 0x7fed90142520 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.916+0000 7fed967f4640 1 --2- 172.21.15.135:0/279317027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c004800 0x7fed900c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.917+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 >> 172.21.15.135:0/279317027 conn(0x7fed900d15e0 msgr2=0x7fed900d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.917+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 shutdown_connections 2024-04-17T06:09:45.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:45.917+0000 7fed967f4640 1 -- 172.21.15.135:0/279317027 wait complete. 2024-04-17T06:09:45.926 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:09:46.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.035+0000 7f4f87396640 1 Processor -- start 2024-04-17T06:09:46.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.035+0000 7f4f87396640 1 -- start start 2024-04-17T06:09:46.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f87396640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4f8015ee30 0x7f4f8009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f87396640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f800cd630 0x7f4f800cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f87396640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f8006db80 con 0x7f4f8009bbe0 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f87396640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f4f8006de60 con 0x7f4f8015ee30 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f87396640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4f800cce30 con 0x7f4f800cd630 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f84931640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f4f8009bbe0 0x7f4f8009af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38660/0 (socket says 172.21.15.135:38660) 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f84931640 1 -- 172.21.15.135:0/3635989232 learned_addr learned my addr 172.21.15.135:0/3635989232 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f85933640 1 --2- 172.21.15.135:0/3635989232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f800cd630 0x7f4f800cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f85132640 1 --2- 172.21.15.135:0/3635989232 >> v2:172.21.15.190:3300/0 conn(0x7f4f8015ee30 0x7f4f8009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f77fff640 1 -- 172.21.15.135:0/3635989232 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4223804481 0 0) 0x7f4f8006db80 con 0x7f4f8009bbe0 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.036+0000 7f4f77fff640 1 -- 172.21.15.135:0/3635989232 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f6c003520 con 0x7f4f8009bbe0 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.037+0000 7f4f85933640 1 -- 172.21.15.135:0/3635989232 >> v2:172.21.15.190:3300/0 conn(0x7f4f8015ee30 msgr2=0x7f4f8009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.037+0000 7f4f85933640 1 --2- 172.21.15.135:0/3635989232 >> v2:172.21.15.190:3300/0 conn(0x7f4f8015ee30 0x7f4f8009a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.037+0000 7f4f85933640 1 -- 172.21.15.135:0/3635989232 >> v1:172.21.15.135:6789/0 conn(0x7f4f8009bbe0 legacy=0x7f4f8009af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.037+0000 7f4f85933640 1 -- 172.21.15.135:0/3635989232 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f800cd110 con 0x7f4f800cd630 2024-04-17T06:09:46.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.037+0000 7f4f85933640 1 --2- 172.21.15.135:0/3635989232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f800cd630 0x7f4f800cdaa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009d90 tx=0x7f4f7802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb63c82fba4b3b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f77fff640 1 -- 172.21.15.135:0/3635989232 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78039070 con 0x7f4f800cd630 2024-04-17T06:09:46.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f77fff640 1 -- 172.21.15.135:0/3635989232 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7802cc50 con 0x7f4f800cd630 2024-04-17T06:09:46.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f77fff640 1 -- 172.21.15.135:0/3635989232 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78034440 con 0x7f4f800cd630 2024-04-17T06:09:46.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f87396640 1 -- 172.21.15.135:0/3635989232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f800cd630 msgr2=0x7f4f800cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f87396640 1 --2- 172.21.15.135:0/3635989232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f800cd630 0x7f4f800cdaa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009d90 tx=0x7f4f7802bea0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f87396640 1 -- 172.21.15.135:0/3635989232 shutdown_connections 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f87396640 1 --2- 172.21.15.135:0/3635989232 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f800cd630 0x7f4f800cdaa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f87396640 1 --2- 172.21.15.135:0/3635989232 >> v2:172.21.15.190:3300/0 conn(0x7f4f8015ee30 0x7f4f8009a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f87396640 1 -- 172.21.15.135:0/3635989232 >> 172.21.15.135:0/3635989232 conn(0x7f4f800b41a0 msgr2=0x7f4f800b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f87396640 1 -- 172.21.15.135:0/3635989232 shutdown_connections 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.038+0000 7f4f87396640 1 -- 172.21.15.135:0/3635989232 wait complete. 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.039+0000 7f4f87396640 1 Processor -- start 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.039+0000 7f4f87396640 1 -- start start 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f87396640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f8009bbe0 0x7f4f80161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f87396640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f800cd630 0x7f4f800b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f87396640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f8015ee30 0x7f4f800b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f87396640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8006d910 con 0x7f4f8015ee30 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f87396640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8006d790 con 0x7f4f800cd630 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f87396640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4f8009b640 con 0x7f4f8009bbe0 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f85933640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f8015ee30 0x7f4f800b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f85933640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f8015ee30 0x7f4f800b5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37696/0 (socket says 172.21.15.135:37696) 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f85933640 1 -- 172.21.15.135:0/562567449 learned_addr learned my addr 172.21.15.135:0/562567449 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f85132640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f8009bbe0 0x7f4f80161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f84931640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f800cd630 0x7f4f800b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.040+0000 7f4f85933640 1 -- 172.21.15.135:0/562567449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f8009bbe0 msgr2=0x7f4f80161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f85933640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f8009bbe0 0x7f4f80161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f85933640 1 -- 172.21.15.135:0/562567449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f800cd630 msgr2=0x7f4f800b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f85933640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f800cd630 0x7f4f800b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f85933640 1 -- 172.21.15.135:0/562567449 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f800b5d80 con 0x7f4f8015ee30 2024-04-17T06:09:46.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f85933640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f8015ee30 0x7f4f800b5810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4f7802c4b0 tx=0x7f4f78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f75ffb640 1 -- 172.21.15.135:0/562567449 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78039050 con 0x7f4f8015ee30 2024-04-17T06:09:46.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f75ffb640 1 -- 172.21.15.135:0/562567449 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f78045070 con 0x7f4f8015ee30 2024-04-17T06:09:46.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f75ffb640 1 -- 172.21.15.135:0/562567449 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78009450 con 0x7f4f8015ee30 2024-04-17T06:09:46.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f800b62e0 con 0x7f4f8015ee30 2024-04-17T06:09:46.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.041+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f800b68a0 con 0x7f4f8015ee30 2024-04-17T06:09:46.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.042+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f80074940 con 0x7f4f8015ee30 2024-04-17T06:09:46.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.043+0000 7f4f75ffb640 1 -- 172.21.15.135:0/562567449 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f7803e610 con 0x7f4f8015ee30 2024-04-17T06:09:46.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.043+0000 7f4f75ffb640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4f6406c8f0 0x7f4f6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.043+0000 7f4f75ffb640 1 -- 172.21.15.135:0/562567449 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4f780ae7f0 con 0x7f4f8015ee30 2024-04-17T06:09:46.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.043+0000 7f4f85132640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4f6406c8f0 0x7f4f6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.043+0000 7f4f85132640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4f6406c8f0 0x7f4f6406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009a10 tx=0x7f4f7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.047+0000 7f4f75ffb640 1 -- 172.21.15.135:0/562567449 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f800b5d80 con 0x7f4f8015ee30 2024-04-17T06:09:46.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.220+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4f800c9440 con 0x7f4f8015ee30 2024-04-17T06:09:46.219 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.221+0000 7f4f75ffb640 1 -- 172.21.15.135:0/562567449 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f4f7807c9b0 con 0x7f4f8015ee30 2024-04-17T06:09:46.219 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:46.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4f6406c8f0 msgr2=0x7f4f6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4f6406c8f0 0x7f4f6406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009a10 tx=0x7f4f7c008040 comp rx=0 tx=0).stop 2024-04-17T06:09:46.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f8015ee30 msgr2=0x7f4f800b5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f8015ee30 0x7f4f800b5810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4f7802c4b0 tx=0x7f4f78004a40 comp rx=0 tx=0).stop 2024-04-17T06:09:46.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 shutdown_connections 2024-04-17T06:09:46.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4f6406c8f0 0x7f4f6406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4f8015ee30 0x7f4f800b5810 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f800cd630 0x7f4f800b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 --2- 172.21.15.135:0/562567449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f8009bbe0 0x7f4f80161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 >> 172.21.15.135:0/562567449 conn(0x7f4f800b41a0 msgr2=0x7f4f8006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:46.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 shutdown_connections 2024-04-17T06:09:46.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.223+0000 7f4f87396640 1 -- 172.21.15.135:0/562567449 wait complete. 2024-04-17T06:09:46.232 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:09:46.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.337+0000 7f0d82159640 1 Processor -- start 2024-04-17T06:09:46.335 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.337+0000 7f0d82159640 1 -- start start 2024-04-17T06:09:46.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.337+0000 7f0d82159640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0d7c1601f0 0x7f0d7c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.337+0000 7f0d82159640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 0x7f0d7c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.337+0000 7f0d82159640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d7c0cd790 con 0x7f0d7c0d0300 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d82159640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0d7c0cda70 con 0x7f0d7c1601f0 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d82159640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7c0cdd50 con 0x7f0d7c0cd2b0 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d7b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0d7c1601f0 0x7f0d7c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d7b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0d7c1601f0 0x7f0d7c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37710/0 (socket says 172.21.15.135:37710) 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d7b7fe640 1 -- 172.21.15.135:0/2331995573 learned_addr learned my addr 172.21.15.135:0/2331995573 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d72ffd640 1 --2- 172.21.15.135:0/2331995573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 0x7f0d7c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d7b7fe640 1 -- 172.21.15.135:0/2331995573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 msgr2=0x7f0d7c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d7b7fe640 1 --2- 172.21.15.135:0/2331995573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 0x7f0d7c0ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d7b7fe640 1 -- 172.21.15.135:0/2331995573 >> v1:172.21.15.190:6789/0 conn(0x7f0d7c0d0300 legacy=0x7f0d7c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.338+0000 7f0d7b7fe640 1 -- 172.21.15.135:0/2331995573 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c0ce030 con 0x7f0d7c1601f0 2024-04-17T06:09:46.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d7b7fe640 1 --2- 172.21.15.135:0/2331995573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c0d1340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009870 tx=0x7f0d6c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eda769035d4a5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d7affd640 1 -- 172.21.15.135:0/2331995573 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c034070 con 0x7f0d7c1601f0 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d7affd640 1 -- 172.21.15.135:0/2331995573 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d6c030840 con 0x7f0d7c1601f0 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d7affd640 1 -- 172.21.15.135:0/2331995573 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c030c10 con 0x7f0d7c1601f0 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d82159640 1 -- 172.21.15.135:0/2331995573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 msgr2=0x7f0d7c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d82159640 1 --2- 172.21.15.135:0/2331995573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c0d1340 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009870 tx=0x7f0d6c02b980 comp rx=0 tx=0).stop 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d82159640 1 -- 172.21.15.135:0/2331995573 shutdown_connections 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d82159640 1 --2- 172.21.15.135:0/2331995573 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 0x7f0d7c0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d82159640 1 --2- 172.21.15.135:0/2331995573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c0d1340 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d82159640 1 -- 172.21.15.135:0/2331995573 >> 172.21.15.135:0/2331995573 conn(0x7f0d7c0b4190 msgr2=0x7f0d7c0b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d82159640 1 -- 172.21.15.135:0/2331995573 shutdown_connections 2024-04-17T06:09:46.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.339+0000 7f0d82159640 1 -- 172.21.15.135:0/2331995573 wait complete. 2024-04-17T06:09:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.341+0000 7f0d82159640 1 Processor -- start 2024-04-17T06:09:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.341+0000 7f0d82159640 1 -- start start 2024-04-17T06:09:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.341+0000 7f0d82159640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 0x7f0d7c070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.341+0000 7f0d82159640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c0d0300 0x7f0d7c159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.341+0000 7f0d82159640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d82159640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c09b280 con 0x7f0d7c1601f0 2024-04-17T06:09:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d82159640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c09b100 con 0x7f0d7c0d0300 2024-04-17T06:09:46.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d82159640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7c0b1aa0 con 0x7f0d7c0cd2b0 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37726/0 (socket says 172.21.15.135:37726) 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 -- 172.21.15.135:0/3732332823 learned_addr learned my addr 172.21.15.135:0/3732332823 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7b7fe640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 0x7f0d7c070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d72ffd640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c0d0300 0x7f0d7c159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 -- 172.21.15.135:0/3732332823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 msgr2=0x7f0d7c070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 0x7f0d7c070450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 -- 172.21.15.135:0/3732332823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c0d0300 msgr2=0x7f0d7c159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c0d0300 0x7f0d7c159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 -- 172.21.15.135:0/3732332823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c15bea0 con 0x7f0d7c1601f0 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d7bfff640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c157780 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c7b0 tx=0x7f0d7400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d78ff9640 1 -- 172.21.15.135:0/3732332823 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74004b60 con 0x7f0d7c1601f0 2024-04-17T06:09:46.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d78ff9640 1 -- 172.21.15.135:0/3732332823 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d74004d00 con 0x7f0d7c1601f0 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d78ff9640 1 -- 172.21.15.135:0/3732332823 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400aa90 con 0x7f0d7c1601f0 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c159ff0 con 0x7f0d7c1601f0 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.342+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c0cabd0 con 0x7f0d7c1601f0 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.343+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7c068a20 con 0x7f0d7c1601f0 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.344+0000 7f0d78ff9640 1 -- 172.21.15.135:0/3732332823 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d7400ac30 con 0x7f0d7c1601f0 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.344+0000 7f0d78ff9640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.344+0000 7f0d78ff9640 1 -- 172.21.15.135:0/3732332823 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0d7408bf90 con 0x7f0d7c1601f0 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.344+0000 7f0d7b7fe640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.345+0000 7f0d7b7fe640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c036000 tx=0x7f0d6c03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.348+0000 7f0d78ff9640 1 -- 172.21.15.135:0/3732332823 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d74020070 con 0x7f0d7c1601f0 2024-04-17T06:09:46.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.562+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d7c0c8e70 con 0x7f0d7c1601f0 2024-04-17T06:09:46.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.562+0000 7f0d78ff9640 1 -- 172.21.15.135:0/3732332823 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f0d74059ac0 con 0x7f0d7c1601f0 2024-04-17T06:09:46.560 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:46.560 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:36.038772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:09:46.561 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 9 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0d5c06c8f0 msgr2=0x7f0d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c036000 tx=0x7f0d6c03b040 comp rx=0 tx=0).stop 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 msgr2=0x7f0d7c157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c157780 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c7b0 tx=0x7f0d7400cc70 comp rx=0 tx=0).stop 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 shutdown_connections 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0d5c06c8f0 0x7f0d5c06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d7c1601f0 0x7f0d7c157780 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d7c0d0300 0x7f0d7c159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 --2- 172.21.15.135:0/3732332823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d7c0cd2b0 0x7f0d7c070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 >> 172.21.15.135:0/3732332823 conn(0x7f0d7c0b4190 msgr2=0x7f0d7c0717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 shutdown_connections 2024-04-17T06:09:46.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.564+0000 7f0d82159640 1 -- 172.21.15.135:0/3732332823 wait complete. 2024-04-17T06:09:46.574 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:46.676 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.678+0000 7f66c0396640 1 Processor -- start 2024-04-17T06:09:46.677 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.678+0000 7f66c0396640 1 -- start start 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.679+0000 7f66c0396640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66b815ec30 0x7f66b80b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.679+0000 7f66c0396640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f66b80b0da0 0x7f66b806b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.679+0000 7f66c0396640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b806c2d0 con 0x7f66b806bed0 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.679+0000 7f66c0396640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f66b80d0330 con 0x7f66b815ec30 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.679+0000 7f66c0396640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f66b8073040 con 0x7f66b80b0da0 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66be132640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66b815ec30 0x7f66b80b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66be132640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66b815ec30 0x7f66b80b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37736/0 (socket says 172.21.15.135:37736) 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66be132640 1 -- 172.21.15.135:0/3385528957 learned_addr learned my addr 172.21.15.135:0/3385528957 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66be933640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f66b806bed0 0x7f66b80708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38662/0 (socket says 172.21.15.135:38662) 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66bd931640 1 --2- 172.21.15.135:0/3385528957 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b0da0 0x7f66b806b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66be132640 1 -- 172.21.15.135:0/3385528957 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b0da0 msgr2=0x7f66b806b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:46.679 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66be132640 1 --2- 172.21.15.135:0/3385528957 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b0da0 0x7f66b806b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66be132640 1 -- 172.21.15.135:0/3385528957 >> v1:172.21.15.135:6789/0 conn(0x7f66b806bed0 legacy=0x7f66b80708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66be132640 1 -- 172.21.15.135:0/3385528957 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b815f010 con 0x7f66b815ec30 2024-04-17T06:09:46.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.680+0000 7f66bd130640 1 -- 172.21.15.135:0/3385528957 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2674815065 0 0) 0x7f66b806c2d0 con 0x7f66b806bed0 2024-04-17T06:09:46.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66be132640 1 --2- 172.21.15.135:0/3385528957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b80b0860 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66b806c940 tx=0x7f66ac02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74427ed722226ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66bd130640 1 -- 172.21.15.135:0/3385528957 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac02f850 con 0x7f66b815ec30 2024-04-17T06:09:46.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66bd130640 1 -- 172.21.15.135:0/3385528957 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac02f9f0 con 0x7f66b815ec30 2024-04-17T06:09:46.680 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66bd130640 1 -- 172.21.15.135:0/3385528957 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac02fdc0 con 0x7f66b815ec30 2024-04-17T06:09:46.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66c0396640 1 -- 172.21.15.135:0/3385528957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 msgr2=0x7f66b80b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66c0396640 1 --2- 172.21.15.135:0/3385528957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b80b0860 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66b806c940 tx=0x7f66ac02ba30 comp rx=0 tx=0).stop 2024-04-17T06:09:46.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66c0396640 1 -- 172.21.15.135:0/3385528957 shutdown_connections 2024-04-17T06:09:46.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66c0396640 1 --2- 172.21.15.135:0/3385528957 >> v2:172.21.15.190:3300/0 conn(0x7f66b80b0da0 0x7f66b806b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66c0396640 1 --2- 172.21.15.135:0/3385528957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b80b0860 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66c0396640 1 -- 172.21.15.135:0/3385528957 >> 172.21.15.135:0/3385528957 conn(0x7f66b80d15e0 msgr2=0x7f66b80d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66c0396640 1 -- 172.21.15.135:0/3385528957 shutdown_connections 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.681+0000 7f66c0396640 1 -- 172.21.15.135:0/3385528957 wait complete. 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66c0396640 1 Processor -- start 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66c0396640 1 -- start start 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66c0396640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b806bed0 0x7f66b814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66c0396640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b0da0 0x7f66b80caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66c0396640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66c0396640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8073850 con 0x7f66b815ec30 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66c0396640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8073570 con 0x7f66b806bed0 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66c0396640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f66b8073b30 con 0x7f66b80b0da0 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66be933640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66be933640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b815c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37752/0 (socket says 172.21.15.135:37752) 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.683+0000 7f66be933640 1 -- 172.21.15.135:0/3677309443 learned_addr learned my addr 172.21.15.135:0/3677309443 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66bd931640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b0da0 0x7f66b80caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66be132640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b806bed0 0x7f66b814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66be933640 1 -- 172.21.15.135:0/3677309443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b0da0 msgr2=0x7f66b80caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66be933640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b0da0 0x7f66b80caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66be933640 1 -- 172.21.15.135:0/3677309443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b806bed0 msgr2=0x7f66b814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66be933640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b806bed0 0x7f66b814ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66be933640 1 -- 172.21.15.135:0/3677309443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b8157da0 con 0x7f66b815ec30 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66be933640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b815c800 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c700 tx=0x7f66b000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66aaffd640 1 -- 172.21.15.135:0/3677309443 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0018070 con 0x7f66b815ec30 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66aaffd640 1 -- 172.21.15.135:0/3677309443 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b000ce10 con 0x7f66b815ec30 2024-04-17T06:09:46.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b8158080 con 0x7f66b815ec30 2024-04-17T06:09:46.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66aaffd640 1 -- 172.21.15.135:0/3677309443 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000a860 con 0x7f66b815ec30 2024-04-17T06:09:46.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.684+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b8158640 con 0x7f66b815ec30 2024-04-17T06:09:46.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.685+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8068a20 con 0x7f66b815ec30 2024-04-17T06:09:46.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.686+0000 7f66aaffd640 1 -- 172.21.15.135:0/3677309443 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66b001b070 con 0x7f66b815ec30 2024-04-17T06:09:46.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.686+0000 7f66aaffd640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c920 0x7f669c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.686+0000 7f66aaffd640 1 -- 172.21.15.135:0/3677309443 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f66b008b320 con 0x7f66b815ec30 2024-04-17T06:09:46.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.686+0000 7f66be132640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c920 0x7f669c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.687+0000 7f66be132640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c920 0x7f669c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66ac036000 tx=0x7f66ac008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.690+0000 7f66aaffd640 1 -- 172.21.15.135:0/3677309443 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66b0058e50 con 0x7f66b815ec30 2024-04-17T06:09:46.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.867+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66b80b1890 con 0x7f66b815ec30 2024-04-17T06:09:46.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.868+0000 7f66aaffd640 1 -- 172.21.15.135:0/3677309443 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f66b005d150 con 0x7f66b815ec30 2024-04-17T06:09:46.866 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:46.866 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:46.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c920 msgr2=0x7f669c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c920 0x7f669c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66ac036000 tx=0x7f66ac008030 comp rx=0 tx=0).stop 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 msgr2=0x7f66b815c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b815c800 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c700 tx=0x7f66b000cbc0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 shutdown_connections 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c920 0x7f669c06ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815ec30 0x7f66b815c800 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b0da0 0x7f66b80caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 --2- 172.21.15.135:0/3677309443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b806bed0 0x7f66b814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 >> 172.21.15.135:0/3677309443 conn(0x7f66b80d15e0 msgr2=0x7f66b80b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 shutdown_connections 2024-04-17T06:09:46.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.870+0000 7f66c0396640 1 -- 172.21.15.135:0/3677309443 wait complete. 2024-04-17T06:09:46.880 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:09:46.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.984+0000 7fa5085a8640 1 Processor -- start 2024-04-17T06:09:46.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.984+0000 7fa5085a8640 1 -- start start 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.985+0000 7fa5085a8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa50015ec30 0x7fa5000b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.985+0000 7fa5085a8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa50015f010 con 0x7fa5000b0da0 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.985+0000 7fa5085a8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa500073290 con 0x7fa50006c0a0 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.985+0000 7fa5085a8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa5000d0330 con 0x7fa50015ec30 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.985+0000 7fa506344640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa50015ec30 0x7fa5000b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.986+0000 7fa506344640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa50015ec30 0x7fa5000b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37758/0 (socket says 172.21.15.135:37758) 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.986+0000 7fa505b43640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa5000b0da0 0x7fa50006b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38672/0 (socket says 172.21.15.135:38672) 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.986+0000 7fa506344640 1 -- 172.21.15.135:0/4077235630 learned_addr learned my addr 172.21.15.135:0/4077235630 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.986+0000 7fa506344640 1 -- 172.21.15.135:0/4077235630 >> v1:172.21.15.190:6789/0 conn(0x7fa50006c0a0 legacy=0x7fa5000b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.986+0000 7fa506344640 1 -- 172.21.15.135:0/4077235630 >> v1:172.21.15.135:6789/0 conn(0x7fa5000b0da0 legacy=0x7fa50006b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.986+0000 7fa506344640 1 -- 172.21.15.135:0/4077235630 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa500073570 con 0x7fa50015ec30 2024-04-17T06:09:46.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.986+0000 7fa505342640 1 -- 172.21.15.135:0/4077235630 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3499311516 0 0) 0x7fa50015f010 con 0x7fa5000b0da0 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa506344640 1 --2- 172.21.15.135:0/4077235630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa50015ec30 0x7fa5000b0860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009e50 tx=0x7fa4fc02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0826c09c3271c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa505342640 1 -- 172.21.15.135:0/4077235630 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc02cbb0 con 0x7fa50015ec30 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa505342640 1 -- 172.21.15.135:0/4077235630 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc02cd50 con 0x7fa50015ec30 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa505342640 1 -- 172.21.15.135:0/4077235630 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc034610 con 0x7fa50015ec30 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa5085a8640 1 -- 172.21.15.135:0/4077235630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa50015ec30 msgr2=0x7fa5000b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa5085a8640 1 --2- 172.21.15.135:0/4077235630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa50015ec30 0x7fa5000b0860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009e50 tx=0x7fa4fc02c010 comp rx=0 tx=0).stop 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa5085a8640 1 -- 172.21.15.135:0/4077235630 shutdown_connections 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa5085a8640 1 --2- 172.21.15.135:0/4077235630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa50015ec30 0x7fa5000b0860 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa5085a8640 1 -- 172.21.15.135:0/4077235630 >> 172.21.15.135:0/4077235630 conn(0x7fa5000d15e0 msgr2=0x7fa5000d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa5085a8640 1 -- 172.21.15.135:0/4077235630 shutdown_connections 2024-04-17T06:09:46.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.987+0000 7fa5085a8640 1 -- 172.21.15.135:0/4077235630 wait complete. 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.988+0000 7fa5085a8640 1 Processor -- start 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa5085a8640 1 -- start start 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa5085a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50006c0a0 0x7fa5001f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa5085a8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5000b0da0 0x7fa500159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa5085a8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50015ec30 0x7fa5000b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa5085a8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa500073b30 con 0x7fa5000b0da0 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa5085a8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa500073040 con 0x7fa50006c0a0 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa5085a8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa5000b5440 con 0x7fa50015ec30 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa505b43640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5000b0da0 0x7fa500159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa505b43640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5000b0da0 0x7fa500159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37760/0 (socket says 172.21.15.135:37760) 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa505b43640 1 -- 172.21.15.135:0/1822779814 learned_addr learned my addr 172.21.15.135:0/1822779814 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:46.989 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa506b45640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50015ec30 0x7fa5000b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.989+0000 7fa506344640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50006c0a0 0x7fa5001f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa505b43640 1 -- 172.21.15.135:0/1822779814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50015ec30 msgr2=0x7fa5000b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa505b43640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50015ec30 0x7fa5000b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa505b43640 1 -- 172.21.15.135:0/1822779814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50006c0a0 msgr2=0x7fa5001f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa505b43640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50006c0a0 0x7fa5001f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa505b43640 1 -- 172.21.15.135:0/1822779814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50015b380 con 0x7fa5000b0da0 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa505b43640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5000b0da0 0x7fa500159440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c4d0 tx=0x7fa4f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa4f6ffd640 1 -- 172.21.15.135:0/1822779814 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0018070 con 0x7fa5000b0da0 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa4f6ffd640 1 -- 172.21.15.135:0/1822779814 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f0007500 con 0x7fa5000b0da0 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa4f6ffd640 1 -- 172.21.15.135:0/1822779814 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0004a20 con 0x7fa5000b0da0 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa500160f40 con 0x7fa5000b0da0 2024-04-17T06:09:46.990 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.990+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa500161500 con 0x7fa5000b0da0 2024-04-17T06:09:46.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.991+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa500068a20 con 0x7fa5000b0da0 2024-04-17T06:09:46.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.991+0000 7fa4f6ffd640 1 -- 172.21.15.135:0/1822779814 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4f001b070 con 0x7fa5000b0da0 2024-04-17T06:09:46.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.992+0000 7fa4f6ffd640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa4e406c8f0 0x7fa4e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:46.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.992+0000 7fa4f6ffd640 1 -- 172.21.15.135:0/1822779814 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa4f008b0b0 con 0x7fa5000b0da0 2024-04-17T06:09:46.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.992+0000 7fa506344640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa4e406c8f0 0x7fa4e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:46.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.992+0000 7fa506344640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa4e406c8f0 0x7fa4e406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc035000 tx=0x7fa4fc030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:46.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:46.996+0000 7fa4f6ffd640 1 -- 172.21.15.135:0/1822779814 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa50015b380 con 0x7fa5000b0da0 2024-04-17T06:09:47.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.210+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa500159980 con 0x7fa5000b0da0 2024-04-17T06:09:47.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.210+0000 7fa4f6ffd640 1 -- 172.21.15.135:0/1822779814 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fa4f0058be0 con 0x7fa5000b0da0 2024-04-17T06:09:47.210 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:47.210 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:36.038772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:09:47.212 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 9 2024-04-17T06:09:47.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa4e406c8f0 msgr2=0x7fa4e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa4e406c8f0 0x7fa4e406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc035000 tx=0x7fa4fc030000 comp rx=0 tx=0).stop 2024-04-17T06:09:47.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5000b0da0 msgr2=0x7fa500159440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5000b0da0 0x7fa500159440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000c4d0 tx=0x7fa4f000c990 comp rx=0 tx=0).stop 2024-04-17T06:09:47.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 shutdown_connections 2024-04-17T06:09:47.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa50015ec30 0x7fa5000b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa4e406c8f0 0x7fa4e406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5000b0da0 0x7fa500159440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 --2- 172.21.15.135:0/1822779814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa50006c0a0 0x7fa5001f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 >> 172.21.15.135:0/1822779814 conn(0x7fa5000d15e0 msgr2=0x7fa5000b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:47.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 shutdown_connections 2024-04-17T06:09:47.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.212+0000 7fa5085a8640 1 -- 172.21.15.135:0/1822779814 wait complete. 2024-04-17T06:09:47.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T06:09:19.967271+0000', 'modified': '2024-04-17T06:09:36.038772+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/3795516465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6834', 'nonce': 3795516465}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 3795516465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-17T06:09:47.624 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T06:09:47.624 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-17T06:09:47.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.732+0000 7f9948de2640 1 Processor -- start 2024-04-17T06:09:47.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.732+0000 7f9948de2640 1 -- start start 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f9948de2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f994415cc60 0x7f99440d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f9948de2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440d0490 0x7f994409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f9948de2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994415d040 con 0x7f994409af30 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f9948de2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f99440d13c0 con 0x7f994415cc60 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f9948de2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f99440b1d20 con 0x7f99440d0490 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f994259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f994415cc60 0x7f99440d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f994259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f994415cc60 0x7f99440d0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37776/0 (socket says 172.21.15.135:37776) 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f994259c640 1 -- 172.21.15.135:0/1226980312 learned_addr learned my addr 172.21.15.135:0/1226980312 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f9939d9b640 1 --2- 172.21.15.135:0/1226980312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440d0490 0x7f994409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f994259c640 1 -- 172.21.15.135:0/1226980312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440d0490 msgr2=0x7f994409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f994259c640 1 --2- 172.21.15.135:0/1226980312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440d0490 0x7f994409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f994259c640 1 -- 172.21.15.135:0/1226980312 >> v1:172.21.15.135:6789/0 conn(0x7f994409af30 legacy=0x7f99440cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f994259c640 1 -- 172.21.15.135:0/1226980312 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99440b2000 con 0x7f994415cc60 2024-04-17T06:09:47.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.733+0000 7f9939d9b640 1 --2- 172.21.15.135:0/1226980312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440d0490 0x7f994409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:47.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.734+0000 7f994259c640 1 --2- 172.21.15.135:0/1226980312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440d0b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9934009870 tx=0x7f993402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc89c2bd9e7ca592 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:47.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.734+0000 7f9941d9b640 1 -- 172.21.15.135:0/1226980312 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934034070 con 0x7f994415cc60 2024-04-17T06:09:47.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.734+0000 7f9941d9b640 1 -- 172.21.15.135:0/1226980312 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9934030840 con 0x7f994415cc60 2024-04-17T06:09:47.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.734+0000 7f9941d9b640 1 -- 172.21.15.135:0/1226980312 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934030c10 con 0x7f994415cc60 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.735+0000 7f9948de2640 1 -- 172.21.15.135:0/1226980312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 msgr2=0x7f99440d0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.735+0000 7f9948de2640 1 --2- 172.21.15.135:0/1226980312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440d0b90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9934009870 tx=0x7f993402b980 comp rx=0 tx=0).stop 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.735+0000 7f9948de2640 1 -- 172.21.15.135:0/1226980312 shutdown_connections 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.735+0000 7f9948de2640 1 --2- 172.21.15.135:0/1226980312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440d0490 0x7f994409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.735+0000 7f9948de2640 1 --2- 172.21.15.135:0/1226980312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440d0b90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.735+0000 7f9948de2640 1 -- 172.21.15.135:0/1226980312 >> 172.21.15.135:0/1226980312 conn(0x7f99440b41a0 msgr2=0x7f99440b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.735+0000 7f9948de2640 1 -- 172.21.15.135:0/1226980312 shutdown_connections 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.735+0000 7f9948de2640 1 -- 172.21.15.135:0/1226980312 wait complete. 2024-04-17T06:09:47.734 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.736+0000 7f9948de2640 1 Processor -- start 2024-04-17T06:09:47.735 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.736+0000 7f9948de2640 1 -- start start 2024-04-17T06:09:47.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9948de2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f994409af30 0x7f99440c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9948de2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99440d0490 0x7f994415fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9948de2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9948de2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99440d16a0 con 0x7f994415cc60 2024-04-17T06:09:47.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9948de2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f99440d10d0 con 0x7f99440d0490 2024-04-17T06:09:47.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9948de2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f99440d1820 con 0x7f994409af30 2024-04-17T06:09:47.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9942d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f994259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f994409af30 0x7f99440c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9942d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37788/0 (socket says 172.21.15.135:37788) 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9942d9d640 1 -- 172.21.15.135:0/3000566995 learned_addr learned my addr 172.21.15.135:0/3000566995 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.737+0000 7f9939d9b640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99440d0490 0x7f994415fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.738+0000 7f9942d9d640 1 -- 172.21.15.135:0/3000566995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f994409af30 msgr2=0x7f99440c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.738+0000 7f9942d9d640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f994409af30 0x7f99440c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.738+0000 7f9942d9d640 1 -- 172.21.15.135:0/3000566995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99440d0490 msgr2=0x7f994415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.738+0000 7f9942d9d640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99440d0490 0x7f994415fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.738+0000 7f9942d9d640 1 -- 172.21.15.135:0/3000566995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9944160110 con 0x7f994415cc60 2024-04-17T06:09:47.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.738+0000 7f9942d9d640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440b0cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f993c00c6b0 tx=0x7f993c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:47.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.739+0000 7f993b7fe640 1 -- 172.21.15.135:0/3000566995 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c003b70 con 0x7f994415cc60 2024-04-17T06:09:47.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.739+0000 7f993b7fe640 1 -- 172.21.15.135:0/3000566995 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c003d30 con 0x7f994415cc60 2024-04-17T06:09:47.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.739+0000 7f9948de2640 1 -- 172.21.15.135:0/3000566995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99440c9780 con 0x7f994415cc60 2024-04-17T06:09:47.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.739+0000 7f993b7fe640 1 -- 172.21.15.135:0/3000566995 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c005310 con 0x7f994415cc60 2024-04-17T06:09:47.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.739+0000 7f9948de2640 1 -- 172.21.15.135:0/3000566995 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994415d530 con 0x7f994415cc60 2024-04-17T06:09:47.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.740+0000 7f993b7fe640 1 -- 172.21.15.135:0/3000566995 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f993c0054b0 con 0x7f994415cc60 2024-04-17T06:09:47.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.741+0000 7f993b7fe640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f992406c920 0x7f992406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.741+0000 7f993b7fe640 1 -- 172.21.15.135:0/3000566995 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f993c08bc10 con 0x7f994415cc60 2024-04-17T06:09:47.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.741+0000 7f994259c640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f992406c920 0x7f992406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.741+0000 7f994259c640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f992406c920 0x7f992406ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9934036000 tx=0x7f9934008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:47.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.742+0000 7f994b9d0740 1 Processor -- start 2024-04-17T06:09:47.740 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.742+0000 7f994b9d0740 1 -- start start 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7145db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 0x55b9c7175ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71462f0 0x55b9c71760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f994b9d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b9c70b09c0 con 0x55b9c71462f0 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f994b9d0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9c7057650 con 0x55b9c71731f0 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f994b9d0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55b9c70ac980 con 0x55b9c7172e10 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71462f0 0x55b9c71760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71462f0 0x55b9c71760e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37798/0 (socket says 172.21.15.135:37798) 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 -- 172.21.15.135:0/2137963342 learned_addr learned my addr 172.21.15.135:0/2137963342 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f994259c640 1 --2- 172.21.15.135:0/2137963342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 -- 172.21.15.135:0/2137963342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 msgr2=0x55b9c7145db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 --2- 172.21.15.135:0/2137963342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7145db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 -- 172.21.15.135:0/2137963342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 msgr2=0x55b9c7175ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 --2- 172.21.15.135:0/2137963342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 0x55b9c7175ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 -- 172.21.15.135:0/2137963342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c6dc32d0 con 0x55b9c71462f0 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.743+0000 7f9942d9d640 1 --2- 172.21.15.135:0/2137963342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71462f0 0x55b9c71760e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f993c061cb0 tx=0x7f993c059290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cb45df9987530e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f9923fff640 1 -- 172.21.15.135:0/2137963342 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0576c0 con 0x55b9c71462f0 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f9923fff640 1 -- 172.21.15.135:0/2137963342 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c057920 con 0x55b9c71462f0 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f9923fff640 1 -- 172.21.15.135:0/2137963342 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c058a70 con 0x55b9c71462f0 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 -- 172.21.15.135:0/2137963342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71462f0 msgr2=0x55b9c71760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2137963342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71462f0 0x55b9c71760e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f993c061cb0 tx=0x7f993c059290 comp rx=0 tx=0).stop 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 -- 172.21.15.135:0/2137963342 shutdown_connections 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2137963342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71462f0 0x55b9c71760e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2137963342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 0x55b9c7175ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2137963342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7145db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.744 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 -- 172.21.15.135:0/2137963342 >> 172.21.15.135:0/2137963342 conn(0x55b9c7148020 msgr2=0x55b9c7190ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:47.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 -- 172.21.15.135:0/2137963342 shutdown_connections 2024-04-17T06:09:47.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.744+0000 7f994b9d0740 1 -- 172.21.15.135:0/2137963342 wait complete. 2024-04-17T06:09:47.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.745+0000 7f994b9d0740 1 -- start start 2024-04-17T06:09:47.745 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.745+0000 7f994b9d0740 1 Processor -- start 2024-04-17T06:09:47.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c7172e10 0x55b9c7191640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c717ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c717b410 0x55b9c717cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994b9d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b9c714a5c0 con 0x55b9c71731f0 2024-04-17T06:09:47.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994b9d0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9c706aa10 con 0x55b9c7172e10 2024-04-17T06:09:47.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994b9d0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55b9c71499b0 con 0x55b9c717b410 2024-04-17T06:09:47.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994b9d0740 10 client.0 _ll_register_callbacks cb 0x55b9c70a6250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:47.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9939d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c717ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9939d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c717ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37814/0 (socket says 172.21.15.135:37814) 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9939d9b640 1 -- 172.21.15.135:0/1301942134 learned_addr learned my addr 172.21.15.135:0/1301942134 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9942d9d640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c717b410 0x55b9c717cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994259c640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c7172e10 0x55b9c7191640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9939d9b640 1 -- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c717b410 msgr2=0x55b9c717cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9939d9b640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c717b410 0x55b9c717cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9939d9b640 1 -- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c7172e10 msgr2=0x55b9c7191640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9939d9b640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c7172e10 0x55b9c7191640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9939d9b640 1 -- 172.21.15.135:0/1301942134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c6e8d6b0 con 0x55b9c71731f0 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f9942d9d640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c717b410 0x55b9c717cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.746+0000 7f994259c640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c7172e10 0x55b9c7191640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f9939d9b640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c717ca00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f992c009f70 tx=0x7f992c0116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded61181f4da0aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f99217fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:09:47.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f99217fa640 1 -- 172.21.15.135:0/1301942134 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c019020 con 0x55b9c71731f0 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f99217fa640 1 -- 172.21.15.135:0/1301942134 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f992c004440 con 0x55b9c71731f0 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 1 -- 172.21.15.135:0/1301942134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f993c057920 con 0x55b9c71731f0 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f99217fa640 1 -- 172.21.15.135:0/1301942134 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f992c023050 con 0x55b9c71731f0 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 10 client.4869 fetch_fsmap learned FSMap version 9 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 1 -- 172.21.15.135:0/1301942134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b9c7177010 con 0x55b9c71731f0 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f99217fa640 1 -- 172.21.15.135:0/1301942134 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c057920 con 0x55b9c71731f0 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f99217fa640 1 -- 172.21.15.135:0/1301942134 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f992c026020 con 0x55b9c71731f0 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 10 client.4869 fetch_fsmap finished waiting for FSMap version 9 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 10 client.4869 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 20 client.4869 populate_metadata read hostname 'smithi135' 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55b9c7177eb0 0x55b9c71a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 4 client.4869 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994b9d0740 1 -- 172.21.15.135:0/1301942134 --> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c70be3e0 con 0x55b9c7177eb0 2024-04-17T06:09:47.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.747+0000 7f994259c640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55b9c7177eb0 0x55b9c71a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.748+0000 7f994259c640 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55b9c7177eb0 0x55b9c71a1510 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25f20bf9d442999 server_cookie=8c0aefab93d06544 in_seq=0 out_seq=0 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.748+0000 7f99217fa640 10 client.4869 ms_handle_connect on v2:172.21.15.190:6834/3795516465 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.748+0000 7f99217fa640 1 -- 172.21.15.135:0/1301942134 <== mds.0 v2:172.21.15.190:6834/3795516465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9c70be3e0 con 0x55b9c7177eb0 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.748+0000 7f99217fa640 10 client.4869 handle_command_reply: tid=0 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.748+0000 7f994b9d0740 1 client.4869 shutdown 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 -- 172.21.15.135:0/1301942134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 msgr2=0x55b9c717ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c717ca00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f992c009f70 tx=0x7f992c0116f0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 -- 172.21.15.135:0/1301942134 shutdown_connections 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55b9c7177eb0 0x55b9c71a1510 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c717b410 0x55b9c717cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c717ca00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 --2- 172.21.15.135:0/1301942134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c7172e10 0x55b9c7191640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f99217fa640 0 client.4869 ms_handle_reset on v2:172.21.15.190:6834/3795516465 2024-04-17T06:09:47.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 -- 172.21.15.135:0/1301942134 >> 172.21.15.135:0/1301942134 conn(0x55b9c71462f0 msgr2=0x55b9c7175b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:47.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 -- 172.21.15.135:0/1301942134 shutdown_connections 2024-04-17T06:09:47.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 1 -- 172.21.15.135:0/1301942134 wait complete. 2024-04-17T06:09:47.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.749+0000 7f994b9d0740 20 client.4869 trim_cache size 0 max 16384 2024-04-17T06:09:47.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.767+0000 7f994b9d0740 1 Processor -- start 2024-04-17T06:09:47.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.767+0000 7f994b9d0740 1 -- start start 2024-04-17T06:09:47.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.767+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7175bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.767+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 0x55b9c7192270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.767+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c717b410 0x55b9c717a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.767+0000 7f994b9d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9944201610 con 0x55b9c717b410 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.767+0000 7f994b9d0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9944200970 con 0x55b9c71731f0 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.767+0000 7f994b9d0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9944200fc0 con 0x55b9c7172e10 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c717b410 0x55b9c717a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c717b410 0x55b9c717a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37822/0 (socket says 172.21.15.135:37822) 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 -- 172.21.15.135:0/2376678547 learned_addr learned my addr 172.21.15.135:0/2376678547 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f994259c640 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7175bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 -- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 msgr2=0x55b9c7175bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9939d9b640 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 0x55b9c7192270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7175bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 -- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 msgr2=0x55b9c7192270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 0x55b9c7192270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 -- 172.21.15.135:0/2376678547 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c7191730 con 0x55b9c717b410 2024-04-17T06:09:47.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f994259c640 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7175bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9939d9b640 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 0x55b9c7192270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f9942d9d640 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c717b410 0x55b9c717a860 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f993c089dd0 tx=0x7f993c070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb451170f9ee9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f99237fe640 1 -- 172.21.15.135:0/2376678547 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0621e0 con 0x55b9c717b410 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f99237fe640 1 -- 172.21.15.135:0/2376678547 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c075070 con 0x55b9c717b410 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.768+0000 7f99237fe640 1 -- 172.21.15.135:0/2376678547 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c057920 con 0x55b9c717b410 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 -- 172.21.15.135:0/2376678547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c717b410 msgr2=0x55b9c717a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c717b410 0x55b9c717a860 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f993c089dd0 tx=0x7f993c070000 comp rx=0 tx=0).stop 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 -- 172.21.15.135:0/2376678547 shutdown_connections 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c717b410 0x55b9c717a860 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c71731f0 0x55b9c7192270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2376678547 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c7175bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 -- 172.21.15.135:0/2376678547 >> 172.21.15.135:0/2376678547 conn(0x55b9c71462f0 msgr2=0x55b9c71413e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 -- 172.21.15.135:0/2376678547 shutdown_connections 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 -- 172.21.15.135:0/2376678547 wait complete. 2024-04-17T06:09:47.769 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 -- start start 2024-04-17T06:09:47.770 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.769+0000 7f994b9d0740 1 Processor -- start 2024-04-17T06:09:47.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c71938c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c71921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f994b9d0740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c717b410 0x55b9c7192710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f994b9d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b9c7177d30 con 0x55b9c71731f0 2024-04-17T06:09:47.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f994b9d0740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55b9c71773b0 con 0x55b9c717b410 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f994b9d0740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55b9c7178350 con 0x55b9c7172e10 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f994b9d0740 10 client.0 _ll_register_callbacks cb 0x7f992c009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f9939d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c71921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f9939d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c71921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37828/0 (socket says 172.21.15.135:37828) 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f9939d9b640 1 -- 172.21.15.135:0/2132757194 learned_addr learned my addr 172.21.15.135:0/2132757194 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f994259c640 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c71938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.770+0000 7f9942d9d640 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c717b410 0x55b9c7192710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f9939d9b640 1 -- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 msgr2=0x55b9c71938c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f9939d9b640 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c71938c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f9939d9b640 1 -- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c717b410 msgr2=0x55b9c7192710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f9939d9b640 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c717b410 0x55b9c7192710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f9939d9b640 1 -- 172.21.15.135:0/2132757194 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c720c680 con 0x55b9c71731f0 2024-04-17T06:09:47.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f9939d9b640 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c71921d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f992c000c00 tx=0x7f992c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68521f43adbe4743 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f99217fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f99217fa640 1 -- 172.21.15.135:0/2132757194 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c019050 con 0x55b9c71731f0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f99217fa640 1 -- 172.21.15.135:0/2132757194 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f992c01e920 con 0x55b9c71731f0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 1 -- 172.21.15.135:0/2132757194 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f993c075070 con 0x55b9c71731f0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f99217fa640 1 -- 172.21.15.135:0/2132757194 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f992c038050 con 0x55b9c71731f0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 10 client.4875 fetch_fsmap learned FSMap version 9 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 1 -- 172.21.15.135:0/2132757194 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b9c720be80 con 0x55b9c71731f0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f99217fa640 1 -- 172.21.15.135:0/2132757194 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c075070 con 0x55b9c71731f0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f99217fa640 1 -- 172.21.15.135:0/2132757194 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f992c03b020 con 0x55b9c71731f0 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 10 client.4875 fetch_fsmap finished waiting for FSMap version 9 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 10 client.4875 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 20 client.4875 populate_metadata read hostname 'smithi135' 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55b9c71b3590 0x55b9c71b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 4 client.4875 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:09:47.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.771+0000 7f994b9d0740 1 -- 172.21.15.135:0/2132757194 --> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] -- command(tid 0: {"prefix": "status"}) -- 0x55b9c70b8250 con 0x55b9c71b3590 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994259c640 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55b9c71b3590 0x55b9c71b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f99217fa640 10 client.4875 ms_handle_connect on v2:172.21.15.190:6834/3795516465 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994259c640 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55b9c71b3590 0x55b9c71b3970 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca55659e2fb630d server_cookie=ee38e7d27b1780d7 in_seq=0 out_seq=0 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f99217fa640 1 -- 172.21.15.135:0/2132757194 <== mds.0 v2:172.21.15.190:6834/3795516465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55b9c70b8250 con 0x55b9c71b3590 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f99217fa640 10 client.4875 handle_command_reply: tid=0 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 client.4875 shutdown 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 -- 172.21.15.135:0/2132757194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 msgr2=0x55b9c71921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c71921d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f992c000c00 tx=0x7f992c008040 comp rx=0 tx=0).stop 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 -- 172.21.15.135:0/2132757194 shutdown_connections 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:6834/3795516465,v1:172.21.15.190:6835/3795516465] conn(0x55b9c71b3590 0x55b9c71b3970 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9c717b410 0x55b9c7192710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9c71731f0 0x55b9c71921d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 --2- 172.21.15.135:0/2132757194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9c7172e10 0x55b9c71938c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f99217fa640 0 client.4875 ms_handle_reset on v2:172.21.15.190:6834/3795516465 2024-04-17T06:09:47.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 -- 172.21.15.135:0/2132757194 >> 172.21.15.135:0/2132757194 conn(0x55b9c71462f0 msgr2=0x55b9c7144cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 -- 172.21.15.135:0/2132757194 shutdown_connections 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.772+0000 7f994b9d0740 1 -- 172.21.15.135:0/2132757194 wait complete. 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f994b9d0740 20 client.4875 trim_cache size 0 max 16384 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster_fsid": "96728bf8-42ba-4aa8-88ba-489677bf3f41", 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stdout: "whoami": 0, 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stdout: "id": 4285, 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stdout: "want_state": "up:active", 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stdout: "state": "up:active", 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "cephfs", 2024-04-17T06:09:47.777 INFO:teuthology.orchestra.run.smithi135.stdout: "rank_uptime": 27.140649133, 2024-04-17T06:09:47.778 INFO:teuthology.orchestra.run.smithi135.stdout: "mdsmap_epoch": 9, 2024-04-17T06:09:47.778 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch": 24, 2024-04-17T06:09:47.778 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch_barrier": 24, 2024-04-17T06:09:47.778 INFO:teuthology.orchestra.run.smithi135.stdout: "uptime": 34.181260254999998 2024-04-17T06:09:47.778 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 -- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f992406c920 msgr2=0x7f992406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f992406c920 0x7f992406ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9934036000 tx=0x7f9934008030 comp rx=0 tx=0).stop 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 -- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 msgr2=0x7f99440b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440b0cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f993c00c6b0 tx=0x7f993c00cb70 comp rx=0 tx=0).stop 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 -- 172.21.15.135:0/3000566995 shutdown_connections 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f992406c920 0x7f992406ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f994415cc60 0x7f99440b0cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99440d0490 0x7f994415fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 --2- 172.21.15.135:0/3000566995 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f994409af30 0x7f99440c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 -- 172.21.15.135:0/3000566995 >> 172.21.15.135:0/3000566995 conn(0x7f99440b41a0 msgr2=0x7f9944070150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 -- 172.21.15.135:0/3000566995 shutdown_connections 2024-04-17T06:09:47.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:47.773+0000 7f9948de2640 1 -- 172.21.15.135:0/3000566995 wait complete. 2024-04-17T06:09:47.782 INFO:teuthology.run_tasks:Running task sequential... 2024-04-17T06:09:47.792 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-17T06:09:47.792 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-17T06:09:47.793 INFO:teuthology.task.print:Enabling mgr modules 2024-04-17T06:09:47.793 INFO:teuthology.run_tasks:Running task kclient... 2024-04-17T06:09:47.806 INFO:tasks.kclient:Mounting kernel clients... 2024-04-17T06:09:47.806 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-17T06:09:47.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:47.806 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-17T06:09:47.823 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-17T06:09:47.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:47.823 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-17T06:09:47.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:47.892 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2024-04-17T06:09:47.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:47.992 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-17T06:09:48.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:48.008 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link delete ceph-brx 2024-04-17T06:09:48.078 INFO:teuthology.orchestra.run.smithi135.stderr:Cannot find device "ceph-brx" 2024-04-17T06:09:48.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:09:48.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:48.080 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T06:09:48.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:48.100 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-04-17T06:09:48.176 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-04-17T06:09:48.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:09:48.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:48.178 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T06:09:48.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:48.236 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2024-04-17T06:09:48.308 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2024-04-17T06:09:48.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-17T06:09:48.310 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-17T06:09:48.310 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T06:09:48.310 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T06:09:48.310 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T06:09:48.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.416+0000 7fd0d8950640 1 Processor -- start 2024-04-17T06:09:48.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.416+0000 7fd0d8950640 1 -- start start 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d8950640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd0d4145010 0x7fd0d40ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d8950640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b5870 0x7fd0d40b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d8950640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d41453f0 con 0x7fd0d40b18a0 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d8950640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd0d40ce150 con 0x7fd0d4145010 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d8950640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d40cd540 con 0x7fd0d40b5870 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd0d4145010 0x7fd0d40ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d2d9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd0d40b18a0 0x7fd0d415bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38688/0 (socket says 172.21.15.135:38688) 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d259c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd0d4145010 0x7fd0d40ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37832/0 (socket says 172.21.15.135:37832) 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d2d9d640 1 -- 172.21.15.135:0/382056636 learned_addr learned my addr 172.21.15.135:0/382056636 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0c9d9b640 1 --2- 172.21.15.135:0/382056636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b5870 0x7fd0d40b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d259c640 1 -- 172.21.15.135:0/382056636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b5870 msgr2=0x7fd0d40b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:48.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d259c640 1 --2- 172.21.15.135:0/382056636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b5870 0x7fd0d40b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d259c640 1 -- 172.21.15.135:0/382056636 >> v1:172.21.15.135:6789/0 conn(0x7fd0d40b18a0 legacy=0x7fd0d415bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d259c640 1 -- 172.21.15.135:0/382056636 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d40cd820 con 0x7fd0d4145010 2024-04-17T06:09:48.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.417+0000 7fd0d1d9b640 1 -- 172.21.15.135:0/382056636 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358860306 0 0) 0x7fd0d41453f0 con 0x7fd0d40b18a0 2024-04-17T06:09:48.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d259c640 1 --2- 172.21.15.135:0/382056636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d40ce5e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400ba40 tx=0x7fd0c402e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3257a6da67b8c199 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d1d9b640 1 -- 172.21.15.135:0/382056636 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c403b070 con 0x7fd0d4145010 2024-04-17T06:09:48.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d1d9b640 1 -- 172.21.15.135:0/382056636 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c402ee30 con 0x7fd0d4145010 2024-04-17T06:09:48.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d1d9b640 1 -- 172.21.15.135:0/382056636 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4036640 con 0x7fd0d4145010 2024-04-17T06:09:48.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d8950640 1 -- 172.21.15.135:0/382056636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 msgr2=0x7fd0d40ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d8950640 1 --2- 172.21.15.135:0/382056636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d40ce5e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400ba40 tx=0x7fd0c402e080 comp rx=0 tx=0).stop 2024-04-17T06:09:48.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d8950640 1 -- 172.21.15.135:0/382056636 shutdown_connections 2024-04-17T06:09:48.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d8950640 1 --2- 172.21.15.135:0/382056636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b5870 0x7fd0d40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d8950640 1 --2- 172.21.15.135:0/382056636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d40ce5e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.418+0000 7fd0d8950640 1 -- 172.21.15.135:0/382056636 >> 172.21.15.135:0/382056636 conn(0x7fd0d406b990 msgr2=0x7fd0d406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:48.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.419+0000 7fd0d8950640 1 -- 172.21.15.135:0/382056636 shutdown_connections 2024-04-17T06:09:48.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.419+0000 7fd0d8950640 1 -- 172.21.15.135:0/382056636 wait complete. 2024-04-17T06:09:48.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.420+0000 7fd0d8950640 1 Processor -- start 2024-04-17T06:09:48.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.420+0000 7fd0d8950640 1 -- start start 2024-04-17T06:09:48.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d8950640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b18a0 0x7fd0d415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d8950640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d40b5870 0x7fd0d41e2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d8950640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d41e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d8950640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40ceb20 con 0x7fd0d4145010 2024-04-17T06:09:48.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d8950640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d40cdde0 con 0x7fd0d40b5870 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d8950640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d40b1d30 con 0x7fd0d40b18a0 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d41e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d41e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37838/0 (socket says 172.21.15.135:37838) 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 -- 172.21.15.135:0/1591462842 learned_addr learned my addr 172.21.15.135:0/1591462842 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d259c640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b18a0 0x7fd0d415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0c9d9b640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d40b5870 0x7fd0d41e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 -- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b18a0 msgr2=0x7fd0d415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b18a0 0x7fd0d415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 -- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d40b5870 msgr2=0x7fd0d41e2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d40b5870 0x7fd0d41e2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 -- 172.21.15.135:0/1591462842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d413e600 con 0x7fd0d4145010 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d259c640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b18a0 0x7fd0d415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:48.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.421+0000 7fd0d2d9d640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d41e2d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c800 tx=0x7fd0cc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.422+0000 7fd0cb7fe640 1 -- 172.21.15.135:0/1591462842 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc018070 con 0x7fd0d4145010 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.422+0000 7fd0cb7fe640 1 -- 172.21.15.135:0/1591462842 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0cc004d80 con 0x7fd0d4145010 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.422+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d413e8e0 con 0x7fd0d4145010 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.422+0000 7fd0cb7fe640 1 -- 172.21.15.135:0/1591462842 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00a880 con 0x7fd0d4145010 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.422+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d413eea0 con 0x7fd0d4145010 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.423+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d4074940 con 0x7fd0d4145010 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.423+0000 7fd0cb7fe640 1 -- 172.21.15.135:0/1591462842 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0cc01b070 con 0x7fd0d4145010 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.423+0000 7fd0cb7fe640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd0b406c8f0 0x7fd0b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.423+0000 7fd0cb7fe640 1 -- 172.21.15.135:0/1591462842 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd0cc08b410 con 0x7fd0d4145010 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.424+0000 7fd0d259c640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd0b406c8f0 0x7fd0b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:48.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.424+0000 7fd0d259c640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd0b406c8f0 0x7fd0b406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4038000 tx=0x7fd0c4032000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:48.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.428+0000 7fd0cb7fe640 1 -- 172.21.15.135:0/1591462842 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0cc0595c0 con 0x7fd0d4145010 2024-04-17T06:09:48.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.639+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd0d41425e0 con 0x7fd0d4145010 2024-04-17T06:09:48.638 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.640+0000 7fd0cb7fe640 1 -- 172.21.15.135:0/1591462842 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd0cc058ec0 con 0x7fd0d4145010 2024-04-17T06:09:48.638 INFO:teuthology.orchestra.run.smithi135.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.641+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd0b406c8f0 msgr2=0x7fd0b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.641+0000 7fd0d8950640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd0b406c8f0 0x7fd0b406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4038000 tx=0x7fd0c4032000 comp rx=0 tx=0).stop 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.641+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 msgr2=0x7fd0d41e2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.641+0000 7fd0d8950640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d41e2d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c800 tx=0x7fd0cc00ccc0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.642+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 shutdown_connections 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.642+0000 7fd0d8950640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd0b406c8f0 0x7fd0b406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.642+0000 7fd0d8950640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4145010 0x7fd0d41e2d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.642+0000 7fd0d8950640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd0d40b5870 0x7fd0d41e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.642+0000 7fd0d8950640 1 --2- 172.21.15.135:0/1591462842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0d40b18a0 0x7fd0d415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.642+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 >> 172.21.15.135:0/1591462842 conn(0x7fd0d406b990 msgr2=0x7fd0d40b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.642+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 shutdown_connections 2024-04-17T06:09:48.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48.642+0000 7fd0d8950640 1 -- 172.21.15.135:0/1591462842 wait complete. 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T06:09:48.651 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T06:09:48.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:48.651 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2024-04-17T06:09:48.667 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:48.667 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:48.667 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:48.667 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:48.667 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:48.667 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:48.667 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:48.667 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f0 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f1 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42733sec preferred_lft 42733sec 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::ec4:7aff:fe88:8083/64 scope link 2024-04-17T06:09:48.668 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:48.669 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T06:09:48.669 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:48.669 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:48.669 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add name ceph-brx type bridge 2024-04-17T06:09:48.669 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr flush dev ceph-brx 2024-04-17T06:09:48.669 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set ceph-brx up 2024-04-17T06:09:48.669 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T06:09:48.669 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:48.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:48.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:48.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:48.822 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T06:09:48.849 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-04-17T06:09:48.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:48.851 DEBUG:teuthology.orchestra.run.smithi135:> route 2024-04-17T06:09:48.907 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2024-04-17T06:09:48.907 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T06:09:48.908 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T06:09:48.908 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T06:09:48.908 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T06:09:48.908 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:48.908 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:48.908 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T06:09:48.908 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T06:09:48.909 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T06:09:48.909 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:48.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:49.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:49.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:49.048 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-17T06:09:49.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:49.064 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list-id 2024-04-17T06:09:49.120 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:49.120 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:49.121 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T06:09:49.121 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-17T06:09:49.121 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:49.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:49.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:49.245 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-17T06:09:49.245 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:49.246 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:49.246 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-17T06:09:49.246 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T06:09:49.246 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-17T06:09:49.246 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-17T06:09:49.246 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-17T06:09:49.247 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:49.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:49.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:49.380 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:49.380 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:49.380 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.0 up 2024-04-17T06:09:49.380 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T06:09:49.381 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:49.454 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:09:49.456 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:09:49.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:49.465 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:49.465 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T06:09:49.555 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T06:09:49.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:49.555 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T06:09:49.610 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-17T06:09:49.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:49.610 DEBUG:teuthology.orchestra.run.smithi135:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-17T06:09:49.699 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-17T06:09:49.699 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-17T06:09:49.699 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:09:49.809 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-17T06:09:49.809 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-17T06:09:49.809 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:09:49.809 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:09:49.809 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:09:49.828 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-17T06:09:49.828 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-17T06:09:49.828 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:09:49.828 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:09:49.828 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:09:49.828 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790" 2024-04-17T06:09:49.828 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2024-04-17T06:09:49.828 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=96728bf8-42ba-4aa8-88ba-489677bf3f41" will pass to kernel 2024-04-17T06:09:49.829 INFO:teuthology.orchestra.run.smithi135.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-17T06:09:49.829 INFO:teuthology.orchestra.run.smithi135.stdout: You just mounted a file system that supports labels which does not 2024-04-17T06:09:49.829 INFO:teuthology.orchestra.run.smithi135.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-17T06:09:49.829 INFO:teuthology.orchestra.run.smithi135.stdout: applications will generate AVC messages and not be allowed access to 2024-04-17T06:09:49.829 INFO:teuthology.orchestra.run.smithi135.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-17T06:09:49.830 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-17T06:09:49.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:49.830 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T06:09:49.859 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:49.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:49.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:49.959 INFO:teuthology.orchestra.run.smithi135.stdout:client4717 2024-04-17T06:09:49.959 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:09:49.959 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:09:49.975 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:50.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:50.067 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:50.073 INFO:teuthology.orchestra.run.smithi135.stdout:client4717 2024-04-17T06:09:50.073 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:50.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:50.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:50.130 INFO:teuthology.orchestra.run.smithi135.stdout:client4717 2024-04-17T06:09:50.130 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/96728bf8-42ba-4aa8-88ba-489677bf3f41.client4717/status') 2024-04-17T06:09:50.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:50.192 INFO:teuthology.orchestra.run.smithi135.stdout:instance: client.4717 (3)172.21.15.135:0/69207512 2024-04-17T06:09:50.192 INFO:teuthology.orchestra.run.smithi135.stdout:blocklisted: false 2024-04-17T06:09:50.193 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records in 2024-04-17T06:09:50.193 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records out 2024-04-17T06:09:50.193 INFO:teuthology.orchestra.run.smithi135.stderr:69 bytes copied, 2.2748e-05 s, 3.0 MB/s 2024-04-17T06:09:50.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:50.200 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-17T06:09:50.200 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T06:09:50.200 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-17T06:09:50.200 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T06:09:50.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.306+0000 7faf046b2640 1 Processor -- start 2024-04-17T06:09:50.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.306+0000 7faf046b2640 1 -- start start 2024-04-17T06:09:50.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.307+0000 7faf046b2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faefc145010 0x7faefc0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:50.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.307+0000 7faf046b2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faefc0b5870 0x7faefc0b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:50.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.307+0000 7faf046b2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faefc1453f0 con 0x7faefc0b18a0 2024-04-17T06:09:50.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.307+0000 7faf046b2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7faefc0ce150 con 0x7faefc0b5870 2024-04-17T06:09:50.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.307+0000 7faf046b2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7faefc0cd540 con 0x7faefc145010 2024-04-17T06:09:50.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf01c4d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faefc0b5870 0x7faefc0b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf01c4d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7faefc0b5870 0x7faefc0b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37840/0 (socket says 172.21.15.135:37840) 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf02c4f640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7faefc0b18a0 0x7faefc15bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38694/0 (socket says 172.21.15.135:38694) 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf01c4d640 1 -- 172.21.15.135:0/2005995526 learned_addr learned my addr 172.21.15.135:0/2005995526 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf0244e640 1 --2- 172.21.15.135:0/2005995526 >> v2:172.21.15.190:3300/0 conn(0x7faefc145010 0x7faefc0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf01c4d640 1 -- 172.21.15.135:0/2005995526 >> v2:172.21.15.190:3300/0 conn(0x7faefc145010 msgr2=0x7faefc0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf01c4d640 1 --2- 172.21.15.135:0/2005995526 >> v2:172.21.15.190:3300/0 conn(0x7faefc145010 0x7faefc0ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf01c4d640 1 -- 172.21.15.135:0/2005995526 >> v1:172.21.15.135:6789/0 conn(0x7faefc0b18a0 legacy=0x7faefc15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf01c4d640 1 -- 172.21.15.135:0/2005995526 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc0cd820 con 0x7faefc0b5870 2024-04-17T06:09:50.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.308+0000 7faf0144c640 1 -- 172.21.15.135:0/2005995526 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4225185188 0 0) 0x7faefc1453f0 con 0x7faefc0b18a0 2024-04-17T06:09:50.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf01c4d640 1 --2- 172.21.15.135:0/2005995526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc0b5cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faefc0b69e0 tx=0x7faeec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab70b4ae3b101bc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf0144c640 1 -- 172.21.15.135:0/2005995526 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec034070 con 0x7faefc0b5870 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf0144c640 1 -- 172.21.15.135:0/2005995526 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec030860 con 0x7faefc0b5870 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf0144c640 1 -- 172.21.15.135:0/2005995526 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec030c30 con 0x7faefc0b5870 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf046b2640 1 -- 172.21.15.135:0/2005995526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 msgr2=0x7faefc0b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf046b2640 1 --2- 172.21.15.135:0/2005995526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc0b5cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faefc0b69e0 tx=0x7faeec02b9a0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf046b2640 1 -- 172.21.15.135:0/2005995526 shutdown_connections 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf046b2640 1 --2- 172.21.15.135:0/2005995526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc0b5cf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf046b2640 1 --2- 172.21.15.135:0/2005995526 >> v2:172.21.15.190:3300/0 conn(0x7faefc145010 0x7faefc0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf046b2640 1 -- 172.21.15.135:0/2005995526 >> 172.21.15.135:0/2005995526 conn(0x7faefc06b990 msgr2=0x7faefc06bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.309+0000 7faf046b2640 1 -- 172.21.15.135:0/2005995526 shutdown_connections 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.310+0000 7faf046b2640 1 -- 172.21.15.135:0/2005995526 wait complete. 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.311+0000 7faf046b2640 1 Processor -- start 2024-04-17T06:09:50.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.311+0000 7faf046b2640 1 -- start start 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.311+0000 7faf046b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faefc0b18a0 0x7faefc143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.311+0000 7faf046b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.311+0000 7faf046b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faefc145010 0x7faefc156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.311+0000 7faf046b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faefc0ceb20 con 0x7faefc0b5870 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.311+0000 7faf046b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7faefc0cdde0 con 0x7faefc145010 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.311+0000 7faf046b2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7faefc0b1d10 con 0x7faefc0b18a0 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc140de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37848/0 (socket says 172.21.15.135:37848) 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 -- 172.21.15.135:0/453963493 learned_addr learned my addr 172.21.15.135:0/453963493 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf02c4f640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faefc145010 0x7faefc156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:50.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf0244e640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faefc0b18a0 0x7faefc143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 -- 172.21.15.135:0/453963493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faefc0b18a0 msgr2=0x7faefc143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faefc0b18a0 0x7faefc143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 -- 172.21.15.135:0/453963493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faefc145010 msgr2=0x7faefc156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faefc145010 0x7faefc156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 -- 172.21.15.135:0/453963493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc0cc0a0 con 0x7faefc0b5870 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf01c4d640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc140de0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faeec02b9e0 tx=0x7faeec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faef2ffd640 1 -- 172.21.15.135:0/453963493 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec048070 con 0x7faefc0b5870 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faef2ffd640 1 -- 172.21.15.135:0/453963493 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec030660 con 0x7faefc0b5870 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faef2ffd640 1 -- 172.21.15.135:0/453963493 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec034040 con 0x7faefc0b5870 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc0cc340 con 0x7faefc0b5870 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.312+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc0cc900 con 0x7faefc0b5870 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.313+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc074940 con 0x7faefc0b5870 2024-04-17T06:09:50.313 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.314+0000 7faef2ffd640 1 -- 172.21.15.135:0/453963493 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faeec037070 con 0x7faefc0b5870 2024-04-17T06:09:50.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.314+0000 7faef2ffd640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:50.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.314+0000 7faf0244e640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:50.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.314+0000 7faef2ffd640 1 -- 172.21.15.135:0/453963493 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7faeec0adc90 con 0x7faefc0b5870 2024-04-17T06:09:50.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.315+0000 7faf0244e640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faee006c8f0 0x7faee006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faef8006fd0 tx=0x7faef8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:50.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.319+0000 7faef2ffd640 1 -- 172.21.15.135:0/453963493 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faeec07b850 con 0x7faefc0b5870 2024-04-17T06:09:50.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.539+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faefc0ce680 con 0x7faefc0b5870 2024-04-17T06:09:50.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.539+0000 7faef2ffd640 1 -- 172.21.15.135:0/453963493 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7faeec07fb50 con 0x7faefc0b5870 2024-04-17T06:09:50.537 INFO:teuthology.orchestra.run.smithi135.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T06:09:50.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faee006c8f0 msgr2=0x7faee006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:50.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faee006c8f0 0x7faee006edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faef8006fd0 tx=0x7faef8008040 comp rx=0 tx=0).stop 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 msgr2=0x7faefc140de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc140de0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faeec02b9e0 tx=0x7faeec004730 comp rx=0 tx=0).stop 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 shutdown_connections 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faefc145010 0x7faefc156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7faee006c8f0 0x7faee006edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faefc0b5870 0x7faefc140de0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 --2- 172.21.15.135:0/453963493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faefc0b18a0 0x7faefc143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 >> 172.21.15.135:0/453963493 conn(0x7faefc06b990 msgr2=0x7faefc140420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 shutdown_connections 2024-04-17T06:09:50.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50.541+0000 7faf046b2640 1 -- 172.21.15.135:0/453963493 wait complete. 2024-04-17T06:09:50.550 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T06:09:50.550 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T06:09:50.550 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2024-04-17T06:09:50.550 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-17T06:09:50.551 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-17T06:09:50.551 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T06:09:50.551 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T06:09:50.551 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T06:09:50.551 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-17T06:09:50.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:50.551 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2024-04-17T06:09:50.566 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:50.566 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:50.566 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f0 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f1 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:50.567 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42731sec preferred_lft 42731sec 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::ec4:7aff:fe88:8083/64 scope link 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 4e:db:16:ca:7f:a9 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::c2:8aff:fe91:6349/64 scope link 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 4e:db:16:ca:7f:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::4cdb:16ff:feca:7fa9/64 scope link tentative 2024-04-17T06:09:50.568 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:50.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:50.569 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T06:09:50.635 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-04-17T06:09:50.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:50.637 DEBUG:teuthology.orchestra.run.smithi135:> route 2024-04-17T06:09:50.693 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2024-04-17T06:09:50.693 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T06:09:50.693 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T06:09:50.693 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T06:09:50.693 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T06:09:50.693 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:50.693 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:50.693 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T06:09:50.694 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T06:09:50.694 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T06:09:50.694 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:50.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:50.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:50.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:50.832 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-17T06:09:50.847 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T06:09:50.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:50.848 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list-id 2024-04-17T06:09:50.903 INFO:teuthology.orchestra.run.smithi135.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-17T06:09:50.904 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:50.904 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:50.904 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T06:09:50.904 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-17T06:09:50.904 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:50.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:51.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:51.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:51.028 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 7e:c1:7e:83:24:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T06:09:51.056 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T06:09:51.057 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:51.057 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::7cc1:7eff:fe83:24bd/64 scope link tentative 2024-04-17T06:09:51.057 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:51.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:51.058 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-17T06:09:51.126 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:51.126 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:51.126 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 7e:c1:7e:83:24:bd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::7cc1:7eff:fe83:24bd/64 scope link tentative 2024-04-17T06:09:51.127 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:51.128 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-17T06:09:51.128 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:51.128 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:51.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-17T06:09:51.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-17T06:09:51.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-17T06:09:51.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-17T06:09:51.129 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-17T06:09:51.129 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:51.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:51.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:51.299 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:51.299 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:09:51.299 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.1 up 2024-04-17T06:09:51.300 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set dev brx.1 master ceph-brx 2024-04-17T06:09:51.300 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:09:51.349 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:51.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:51.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:51.384 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-17T06:09:51.399 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-17T06:09:51.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:51.400 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-17T06:09:51.455 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-17T06:09:51.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:51.455 DEBUG:teuthology.orchestra.run.smithi135:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-17T06:09:51.542 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-17T06:09:51.542 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-17T06:09:51.543 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:09:51.551 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-17T06:09:51.551 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-17T06:09:51.551 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:09:51.551 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:09:51.551 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:09:51.560 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-17T06:09:51.560 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-17T06:09:51.560 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:09:51.560 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:09:51.560 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:09:51.560 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790" 2024-04-17T06:09:51.560 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2024-04-17T06:09:51.560 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=96728bf8-42ba-4aa8-88ba-489677bf3f41" will pass to kernel 2024-04-17T06:09:51.561 INFO:teuthology.orchestra.run.smithi135.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-17T06:09:51.561 INFO:teuthology.orchestra.run.smithi135.stdout: You just mounted a file system that supports labels which does not 2024-04-17T06:09:51.561 INFO:teuthology.orchestra.run.smithi135.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-17T06:09:51.561 INFO:teuthology.orchestra.run.smithi135.stdout: applications will generate AVC messages and not be allowed access to 2024-04-17T06:09:51.561 INFO:teuthology.orchestra.run.smithi135.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-17T06:09:51.562 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-17T06:09:51.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:51.562 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-17T06:09:51.590 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:51.681 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:51.683 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:51.689 INFO:teuthology.orchestra.run.smithi135.stdout:client4896 2024-04-17T06:09:51.689 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:09:51.689 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:09:51.705 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:51.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:51.797 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:51.803 INFO:teuthology.orchestra.run.smithi135.stdout:client4896 2024-04-17T06:09:51.803 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:51.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:51.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:51.859 INFO:teuthology.orchestra.run.smithi135.stdout:client4896 2024-04-17T06:09:51.859 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/96728bf8-42ba-4aa8-88ba-489677bf3f41.client4896/status') 2024-04-17T06:09:51.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:51.921 INFO:teuthology.orchestra.run.smithi135.stdout:instance: client.4896 (3)192.168.0.2:0/11462720 2024-04-17T06:09:51.922 INFO:teuthology.orchestra.run.smithi135.stdout:blocklisted: false 2024-04-17T06:09:51.922 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records in 2024-04-17T06:09:51.922 INFO:teuthology.orchestra.run.smithi135.stderr:0+1 records out 2024-04-17T06:09:51.922 INFO:teuthology.orchestra.run.smithi135.stderr:67 bytes copied, 2.3892e-05 s, 2.8 MB/s 2024-04-17T06:09:51.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:51.930 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-17T06:09:51.930 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T06:09:51.930 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-17T06:09:51.931 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T06:09:52.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.035+0000 7f174759e640 1 Processor -- start 2024-04-17T06:09:52.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.035+0000 7f174759e640 1 -- start start 2024-04-17T06:09:52.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f174759e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f174815ead0 0x7f17480d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:52.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f174759e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f174806caa0 0x7f17480b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:52.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f174759e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17480d1a10 con 0x7f174806d4b0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f174759e640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f17480cd490 con 0x7f174806caa0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f174759e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f17480ce930 con 0x7f174815ead0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f1746d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f174806caa0 0x7f17480b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f1746d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f174806caa0 0x7f17480b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37850/0 (socket says 172.21.15.135:37850) 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f1746d9d640 1 -- 172.21.15.135:0/110772043 learned_addr learned my addr 172.21.15.135:0/110772043 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.036+0000 7f173dd9b640 1 --2- 172.21.15.135:0/110772043 >> v2:172.21.15.190:3300/0 conn(0x7f174815ead0 0x7f17480d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.037+0000 7f1746d9d640 1 -- 172.21.15.135:0/110772043 >> v2:172.21.15.190:3300/0 conn(0x7f174815ead0 msgr2=0x7f17480d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.037+0000 7f1746d9d640 1 --2- 172.21.15.135:0/110772043 >> v2:172.21.15.190:3300/0 conn(0x7f174815ead0 0x7f17480d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.037+0000 7f1746d9d640 1 -- 172.21.15.135:0/110772043 >> v1:172.21.15.190:6789/0 conn(0x7f174806d4b0 legacy=0x7f174806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.037+0000 7f1746d9d640 1 -- 172.21.15.135:0/110772043 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17480ccb60 con 0x7f174806caa0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.037+0000 7f1746d9d640 1 --2- 172.21.15.135:0/110772043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806caa0 0x7f17480b75f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1748073530 tx=0x7f174002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9364b0b7218b1ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.037+0000 7f1745d9b640 1 -- 172.21.15.135:0/110772043 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174002ca80 con 0x7f174806caa0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.037+0000 7f1745d9b640 1 -- 172.21.15.135:0/110772043 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f174002cc20 con 0x7f174806caa0 2024-04-17T06:09:52.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f1745d9b640 1 -- 172.21.15.135:0/110772043 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17400343f0 con 0x7f174806caa0 2024-04-17T06:09:52.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f174759e640 1 -- 172.21.15.135:0/110772043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806caa0 msgr2=0x7f17480b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f174759e640 1 --2- 172.21.15.135:0/110772043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806caa0 0x7f17480b75f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1748073530 tx=0x7f174002bee0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f174759e640 1 -- 172.21.15.135:0/110772043 shutdown_connections 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f174759e640 1 --2- 172.21.15.135:0/110772043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806caa0 0x7f17480b75f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f174759e640 1 --2- 172.21.15.135:0/110772043 >> v2:172.21.15.190:3300/0 conn(0x7f174815ead0 0x7f17480d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f174759e640 1 -- 172.21.15.135:0/110772043 >> 172.21.15.135:0/110772043 conn(0x7f17480b3f60 msgr2=0x7f17480b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f174759e640 1 -- 172.21.15.135:0/110772043 shutdown_connections 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.038+0000 7f174759e640 1 -- 172.21.15.135:0/110772043 wait complete. 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f174759e640 1 Processor -- start 2024-04-17T06:09:52.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f174759e640 1 -- start start 2024-04-17T06:09:52.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f174759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f174806caa0 0x7f17481529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:52.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f174759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806d4b0 0x7f1748153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:52.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f174759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f174815ead0 0x7f1748158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:52.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f174759e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17480d16b0 con 0x7f174806d4b0 2024-04-17T06:09:52.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f174759e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f17480cd120 con 0x7f174815ead0 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f174759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f17480b52c0 con 0x7f174806caa0 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f173dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806d4b0 0x7f1748153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f173dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806d4b0 0x7f1748153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37862/0 (socket says 172.21.15.135:37862) 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.040+0000 7f173dd9b640 1 -- 172.21.15.135:0/3142177608 learned_addr learned my addr 172.21.15.135:0/3142177608 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f1746d9d640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f174815ead0 0x7f1748158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f174659c640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f174806caa0 0x7f17481529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173dd9b640 1 -- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f174806caa0 msgr2=0x7f17481529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173dd9b640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f174806caa0 0x7f17481529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173dd9b640 1 -- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f174815ead0 msgr2=0x7f1748158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173dd9b640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f174815ead0 0x7f1748158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173dd9b640 1 -- 172.21.15.135:0/3142177608 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1748152f30 con 0x7f174806d4b0 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f174659c640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f174806caa0 0x7f17481529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173dd9b640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806d4b0 0x7f1748153a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f173000d7d0 tx=0x7f173000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:52.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173f7fe640 1 -- 172.21.15.135:0/3142177608 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173000b6c0 con 0x7f174806d4b0 2024-04-17T06:09:52.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173f7fe640 1 -- 172.21.15.135:0/3142177608 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1730019070 con 0x7f174806d4b0 2024-04-17T06:09:52.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17481f5c50 con 0x7f174806d4b0 2024-04-17T06:09:52.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f173f7fe640 1 -- 172.21.15.135:0/3142177608 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173000bac0 con 0x7f174806d4b0 2024-04-17T06:09:52.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.041+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1748154700 con 0x7f174806d4b0 2024-04-17T06:09:52.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.042+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174804fdc0 con 0x7f174806d4b0 2024-04-17T06:09:52.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.043+0000 7f173f7fe640 1 -- 172.21.15.135:0/3142177608 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f173000bc60 con 0x7f174806d4b0 2024-04-17T06:09:52.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.043+0000 7f173f7fe640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f172806c8f0 0x7f172806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:52.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.043+0000 7f174659c640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f172806c8f0 0x7f172806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:52.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.043+0000 7f173f7fe640 1 -- 172.21.15.135:0/3142177608 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f173008c2b0 con 0x7f174806d4b0 2024-04-17T06:09:52.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.044+0000 7f174659c640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f172806c8f0 0x7f172806edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f17480b3950 tx=0x7f1738009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:52.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.048+0000 7f173f7fe640 1 -- 172.21.15.135:0/3142177608 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f173001c360 con 0x7f174806d4b0 2024-04-17T06:09:52.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.259+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1748068a20 con 0x7f174806d4b0 2024-04-17T06:09:52.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.259+0000 7f173f7fe640 1 -- 172.21.15.135:0/3142177608 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f173005a430 con 0x7f174806d4b0 2024-04-17T06:09:52.257 INFO:teuthology.orchestra.run.smithi135.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T06:09:52.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f172806c8f0 msgr2=0x7f172806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:52.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f172806c8f0 0x7f172806edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f17480b3950 tx=0x7f1738009040 comp rx=0 tx=0).stop 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806d4b0 msgr2=0x7f1748153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806d4b0 0x7f1748153a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f173000d7d0 tx=0x7f173000dc90 comp rx=0 tx=0).stop 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 shutdown_connections 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f174815ead0 0x7f1748158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f172806c8f0 0x7f172806edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f174806d4b0 0x7f1748153a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 --2- 172.21.15.135:0/3142177608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f174806caa0 0x7f17481529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 >> 172.21.15.135:0/3142177608 conn(0x7f17480b3f60 msgr2=0x7f17480b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 shutdown_connections 2024-04-17T06:09:52.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:52.261+0000 7f174759e640 1 -- 172.21.15.135:0/3142177608 wait complete. 2024-04-17T06:09:52.270 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T06:09:52.270 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T06:09:52.271 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-04-17T06:09:52.271 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-17T06:09:52.271 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-17T06:09:52.271 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T06:09:52.271 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T06:09:52.271 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T06:09:52.271 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-17T06:09:52.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:52.271 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-04-17T06:09:52.291 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:52.291 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:52.291 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:52.291 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:52.291 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:52.291 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:52.291 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:52.291 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42710sec preferred_lft 42710sec 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-04-17T06:09:52.292 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:52.293 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-17T06:09:52.293 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:52.293 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:52.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2024-04-17T06:09:52.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2024-04-17T06:09:52.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2024-04-17T06:09:52.293 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-17T06:09:52.293 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:52.386 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:52.576 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:52.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:52.583 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T06:09:52.611 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-04-17T06:09:52.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:52.613 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-04-17T06:09:52.684 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-04-17T06:09:52.684 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T06:09:52.684 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T06:09:52.684 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T06:09:52.684 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T06:09:52.684 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:52.684 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:52.684 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T06:09:52.684 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T06:09:52.685 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T06:09:52.685 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:52.774 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:53.011 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:53.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:53.017 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T06:09:53.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:53.034 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-04-17T06:09:53.090 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:53.090 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:53.090 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T06:09:53.091 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-17T06:09:53.091 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:53.181 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:53.212 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:53.220 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-17T06:09:53.220 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:53.220 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:53.220 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-17T06:09:53.220 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-17T06:09:53.220 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-17T06:09:53.221 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-17T06:09:53.221 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-17T06:09:53.221 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:53.270 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:53.359 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:53.365 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:53.365 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:53.365 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-04-17T06:09:53.365 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2024-04-17T06:09:53.365 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:53.414 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:53.443 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:53.449 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:53.450 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-17T06:09:53.472 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-17T06:09:53.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:53.473 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-17T06:09:53.528 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-17T06:09:53.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:53.528 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-17T06:09:53.616 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-17T06:09:53.616 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-17T06:09:53.616 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:09:53.727 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-17T06:09:53.727 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-17T06:09:53.727 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:09:53.727 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:09:53.727 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:09:53.732 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-17T06:09:53.732 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-17T06:09:53.732 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:09:53.732 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:09:53.732 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:09:53.733 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790" 2024-04-17T06:09:53.733 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2024-04-17T06:09:53.733 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=96728bf8-42ba-4aa8-88ba-489677bf3f41" will pass to kernel 2024-04-17T06:09:53.733 INFO:teuthology.orchestra.run.smithi190.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-17T06:09:53.734 INFO:teuthology.orchestra.run.smithi190.stdout: You just mounted a file system that supports labels which does not 2024-04-17T06:09:53.734 INFO:teuthology.orchestra.run.smithi190.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-17T06:09:53.734 INFO:teuthology.orchestra.run.smithi190.stdout: applications will generate AVC messages and not be allowed access to 2024-04-17T06:09:53.734 INFO:teuthology.orchestra.run.smithi190.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-17T06:09:53.735 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-17T06:09:53.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:53.736 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-17T06:09:53.764 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:53.854 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:53.856 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:53.861 INFO:teuthology.orchestra.run.smithi190.stdout:client4905 2024-04-17T06:09:53.861 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:09:53.862 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:09:53.878 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:53.966 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:53.970 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:53.974 INFO:teuthology.orchestra.run.smithi190.stdout:client4905 2024-04-17T06:09:53.974 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:54.024 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:54.026 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:54.032 INFO:teuthology.orchestra.run.smithi190.stdout:client4905 2024-04-17T06:09:54.032 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/96728bf8-42ba-4aa8-88ba-489677bf3f41.client4905/status') 2024-04-17T06:09:54.081 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:54.095 INFO:teuthology.orchestra.run.smithi190.stdout:instance: client.4905 (3)172.21.15.190:0/3901468595 2024-04-17T06:09:54.096 INFO:teuthology.orchestra.run.smithi190.stdout:blocklisted: false 2024-04-17T06:09:54.096 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records in 2024-04-17T06:09:54.096 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records out 2024-04-17T06:09:54.096 INFO:teuthology.orchestra.run.smithi190.stderr:71 bytes copied, 2.252e-05 s, 3.2 MB/s 2024-04-17T06:09:54.097 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:54.103 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-17T06:09:54.103 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-17T06:09:54.103 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-17T06:09:54.103 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T06:09:54.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.220+0000 7f84b8b72640 1 Processor -- start 2024-04-17T06:09:54.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.220+0000 7f84b8b72640 1 -- start start 2024-04-17T06:09:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b8b72640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f84b41632e0 0x7f84b406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b8b72640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b8b72640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b40d1170 con 0x7f84b406bf30 2024-04-17T06:09:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b8b72640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f84b40710c0 con 0x7f84b41632e0 2024-04-17T06:09:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b8b72640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f84b4162e50 con 0x7f84b40706f0 2024-04-17T06:09:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b259c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f84b406bf30 0x7f84b40d06d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:38700/0 (socket says 172.21.15.135:38700) 2024-04-17T06:09:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b259c640 1 -- 172.21.15.135:0/2936847732 learned_addr learned my addr 172.21.15.135:0/2936847732 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:54.220 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b2d9d640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b1d9b640 1 --2- 172.21.15.135:0/2936847732 >> v2:172.21.15.190:3300/0 conn(0x7f84b41632e0 0x7f84b406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1863486727 0 0) 0x7f84b40d1170 con 0x7f84b406bf30 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f849c003520 con 0x7f84b406bf30 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1059251472 0 0) 0x7f849c003520 con 0x7f84b406bf30 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 msgr2=0x7f84b4070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4070b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 >> v2:172.21.15.190:3300/0 conn(0x7f84b41632e0 msgr2=0x7f84b406c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 --2- 172.21.15.135:0/2936847732 >> v2:172.21.15.190:3300/0 conn(0x7f84b41632e0 0x7f84b406c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b4162800 con 0x7f84b406bf30 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f84a8001670 con 0x7f84b406bf30 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 >> v1:172.21.15.135:6789/0 conn(0x7f84b406bf30 legacy=0x7f84b40d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f849c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.221 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849c003f30 0x7f849c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f849c004d40 0x7f849c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84b40710c0 con 0x7f849c004d40 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4162e50 con 0x7f849c003f30 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.221+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f849c00b9f0 con 0x7f84b40706f0 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b259c640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f849c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b2d9d640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f849c004d40 0x7f849c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b1d9b640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849c003f30 0x7f849c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b259c640 1 -- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849c003f30 msgr2=0x7f849c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b259c640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f849c003f30 0x7f849c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b259c640 1 -- 172.21.15.135:0/2936847732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f849c004d40 msgr2=0x7f849c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b259c640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f849c004d40 0x7f849c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b259c640 1 -- 172.21.15.135:0/2936847732 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c003520 con 0x7f84b40706f0 2024-04-17T06:09:54.222 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.222+0000 7f84b2d9d640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f849c004d40 0x7f849c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:54.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b259c640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f849c0039b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001050 tx=0x7f849c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936cc9fbc8d0b99c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8007d70 con 0x7f84b40706f0 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a8030070 con 0x7f84b40706f0 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b159a640 1 -- 172.21.15.135:0/2936847732 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a802cd80 con 0x7f84b40706f0 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b8b72640 1 -- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 msgr2=0x7f849c0039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b8b72640 1 --2- 172.21.15.135:0/2936847732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f849c0039b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001050 tx=0x7f849c005830 comp rx=0 tx=0).stop 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b2d9d640 1 -- 172.21.15.135:0/2936847732 reap_dead start 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b8b72640 1 -- 172.21.15.135:0/2936847732 shutdown_connections 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b8b72640 1 -- 172.21.15.135:0/2936847732 >> 172.21.15.135:0/2936847732 conn(0x7f84b409a9f0 msgr2=0x7f84b40b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b8b72640 1 -- 172.21.15.135:0/2936847732 shutdown_connections 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.223+0000 7f84b8b72640 1 -- 172.21.15.135:0/2936847732 wait complete. 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.225+0000 7f84b8b72640 1 Processor -- start 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.225+0000 7f84b8b72640 1 -- start start 2024-04-17T06:09:54.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b8b72640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b8b72640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b40cbbe0 0x7f84b40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b8b72640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84b40c8c50 0x7f84b40ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b8b72640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4070b80 con 0x7f84b40c8c50 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b8b72640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f84b40d0de0 con 0x7f84b40cbbe0 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b8b72640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f84b4070d00 con 0x7f84b40706f0 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84b40c8c50 0x7f84b40ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b1d9b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b40cbbe0 0x7f84b40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:34878/0 (socket says 172.21.15.135:34878) 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 -- 172.21.15.135:0/3668650965 learned_addr learned my addr 172.21.15.135:0/3668650965 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84b40c8c50 0x7f84b40ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37872/0 (socket says 172.21.15.135:37872) 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b1d9b640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b40cbbe0 0x7f84b40c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:59980/0 (socket says 172.21.15.135:59980) 2024-04-17T06:09:54.227 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 -- 172.21.15.135:0/3668650965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b40cbbe0 msgr2=0x7f84b40c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b40cbbe0 0x7f84b40c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 -- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84b40c8c50 msgr2=0x7f84b40ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84b40c8c50 0x7f84b40ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 -- 172.21.15.135:0/3668650965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b40cc070 con 0x7f84b40706f0 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b2d9d640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84b40c8c50 0x7f84b40ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b259c640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4156da0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84a802c590 tx=0x7f84a8000fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84a2ffd640 1 -- 172.21.15.135:0/3668650965 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004bf0 con 0x7f84b40706f0 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84a2ffd640 1 -- 172.21.15.135:0/3668650965 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a8004d90 con 0x7f84b40706f0 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84a2ffd640 1 -- 172.21.15.135:0/3668650965 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8007270 con 0x7f84b40706f0 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.226+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b40cc350 con 0x7f84b40706f0 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.227+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b40cc910 con 0x7f84b40706f0 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.227+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b4074940 con 0x7f84b40706f0 2024-04-17T06:09:54.228 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.228+0000 7f84a2ffd640 1 -- 172.21.15.135:0/3668650965 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84a8047020 con 0x7f84b40706f0 2024-04-17T06:09:54.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.228+0000 7f84a2ffd640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f849406c920 0x7f849406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:54.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.228+0000 7f84a2ffd640 1 -- 172.21.15.135:0/3668650965 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f84a80acfc0 con 0x7f84b40706f0 2024-04-17T06:09:54.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.229+0000 7f84b1d9b640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f849406c920 0x7f849406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:54.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.229+0000 7f84b1d9b640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f849406c920 0x7f849406ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f84b40c7ff0 tx=0x7f84a400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:54.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.233+0000 7f84a2ffd640 1 -- 172.21.15.135:0/3668650965 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84a807b0e0 con 0x7f84b40706f0 2024-04-17T06:09:54.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.444+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84b4073bc0 con 0x7f84b40706f0 2024-04-17T06:09:54.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.444+0000 7f84a2ffd640 1 -- 172.21.15.135:0/3668650965 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f84a807a9e0 con 0x7f84b40706f0 2024-04-17T06:09:54.442 INFO:teuthology.orchestra.run.smithi135.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f849406c920 msgr2=0x7f849406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f849406c920 0x7f849406ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f84b40c7ff0 tx=0x7f84a400b040 comp rx=0 tx=0).stop 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 msgr2=0x7f84b4156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4156da0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84a802c590 tx=0x7f84a8000fe0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 shutdown_connections 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f849406c920 0x7f849406ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84b40c8c50 0x7f84b40ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84b40cbbe0 0x7f84b40c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 --2- 172.21.15.135:0/3668650965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84b40706f0 0x7f84b4156da0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 >> 172.21.15.135:0/3668650965 conn(0x7f84b409a9f0 msgr2=0x7f84b40d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:54.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 shutdown_connections 2024-04-17T06:09:54.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:54.446+0000 7f84b8b72640 1 -- 172.21.15.135:0/3668650965 wait complete. 2024-04-17T06:09:54.455 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-17T06:09:54.455 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T06:09:54.456 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-04-17T06:09:54.456 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-17T06:09:54.456 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-17T06:09:54.456 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-17T06:09:54.456 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-17T06:09:54.456 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T06:09:54.456 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-17T06:09:54.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:54.456 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-04-17T06:09:54.473 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T06:09:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42708sec preferred_lft 42708sec 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 66:92:72:a8:56:7f brd ff:ff:ff:ff:ff:ff 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::4857:30ff:fe97:155/64 scope link tentative 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 66:92:72:a8:56:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::6492:72ff:fea8:567f/64 scope link tentative 2024-04-17T06:09:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:54.476 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T06:09:54.542 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-04-17T06:09:54.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:54.544 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-04-17T06:09:54.601 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-04-17T06:09:54.601 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T06:09:54.601 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T06:09:54.601 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T06:09:54.601 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T06:09:54.601 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:54.601 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:54.602 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T06:09:54.602 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T06:09:54.602 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T06:09:54.602 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:54.692 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:54.736 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:54.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:54.743 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T06:09:54.759 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-17T06:09:54.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:54.759 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2024-04-17T06:09:54.815 INFO:teuthology.orchestra.run.smithi190.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-17T06:09:54.815 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:54.816 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:54.816 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T06:09:54.816 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-17T06:09:54.816 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:54.904 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:54.933 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:54.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:54.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 5a:18:23:d2:a1:1a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::5818:23ff:fed2:a11a/64 scope link 2024-04-17T06:09:54.968 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:54.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:54.969 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 5a:18:23:d2:a1:1a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-17T06:09:55.038 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:55.039 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::5818:23ff:fed2:a11a/64 scope link 2024-04-17T06:09:55.039 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:09:55.040 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-17T06:09:55.040 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:55.040 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:55.040 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-17T06:09:55.040 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-17T06:09:55.040 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-17T06:09:55.040 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-17T06:09:55.040 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-17T06:09:55.040 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:55.128 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:55.204 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:55.211 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:09:55.211 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:09:55.211 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.1 up 2024-04-17T06:09:55.211 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.1 master ceph-brx 2024-04-17T06:09:55.211 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:09:55.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:55.294 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:55.300 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:55.300 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-17T06:09:55.315 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-17T06:09:55.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:09:55.316 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-17T06:09:55.371 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-17T06:09:55.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:55.371 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-17T06:09:55.457 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-17T06:09:55.457 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-17T06:09:55.457 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:09:55.466 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-17T06:09:55.466 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-17T06:09:55.466 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:09:55.466 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:09:55.466 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:09:55.471 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-17T06:09:55.471 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-17T06:09:55.471 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790" 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=96728bf8-42ba-4aa8-88ba-489677bf3f41" will pass to kernel 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout: You just mounted a file system that supports labels which does not 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout: applications will generate AVC messages and not be allowed access to 2024-04-17T06:09:55.472 INFO:teuthology.orchestra.run.smithi190.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-17T06:09:55.474 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-17T06:09:55.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:09:55.474 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-17T06:09:55.503 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:55.592 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:55.593 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:55.602 INFO:teuthology.orchestra.run.smithi190.stdout:client4914 2024-04-17T06:09:55.602 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:09:55.603 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:09:55.619 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:55.708 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:55.709 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:55.715 INFO:teuthology.orchestra.run.smithi190.stdout:client4914 2024-04-17T06:09:55.715 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:09:55.764 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:09:55.766 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:55.771 INFO:teuthology.orchestra.run.smithi190.stdout:client4914 2024-04-17T06:09:55.771 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/96728bf8-42ba-4aa8-88ba-489677bf3f41.client4914/status') 2024-04-17T06:09:55.820 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:09:55.834 INFO:teuthology.orchestra.run.smithi190.stdout:instance: client.4914 (3)172.21.15.190:0/768945999 2024-04-17T06:09:55.834 INFO:teuthology.orchestra.run.smithi190.stdout:blocklisted: false 2024-04-17T06:09:55.834 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records in 2024-04-17T06:09:55.834 INFO:teuthology.orchestra.run.smithi190.stderr:0+1 records out 2024-04-17T06:09:55.834 INFO:teuthology.orchestra.run.smithi190.stderr:70 bytes copied, 2.322e-05 s, 3.0 MB/s 2024-04-17T06:09:55.835 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:09:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:09:55.841 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-17T06:09:55.858 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:55.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.975+0000 7f311a6eb640 1 Processor -- start 2024-04-17T06:09:55.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.975+0000 7f311a6eb640 1 -- start start 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.976+0000 7f311a6eb640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3114158a50 0x7f311415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.976+0000 7f311a6eb640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f311415fb80 con 0x7f3114158e30 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.976+0000 7f311a6eb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f311415f2e0 con 0x7f311406f610 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.976+0000 7f311a6eb640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f311415b840 con 0x7f3114158a50 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.976+0000 7f3113fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3114158a50 0x7f311415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.976+0000 7f3113fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3114158a50 0x7f311415ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54824/0 (socket says 172.21.15.135:54824) 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.976+0000 7f31137fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3114158e30 0x7f31140b3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51624/0 (socket says 172.21.15.135:51624) 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.976+0000 7f3113fff640 1 -- 172.21.15.135:0/814317887 learned_addr learned my addr 172.21.15.135:0/814317887 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:55.975 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.977+0000 7f3113fff640 1 -- 172.21.15.135:0/814317887 >> v1:172.21.15.190:6789/0 conn(0x7f311406f610 legacy=0x7f311415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:55.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.977+0000 7f3113fff640 1 -- 172.21.15.135:0/814317887 >> v1:172.21.15.135:6789/0 conn(0x7f3114158e30 legacy=0x7f31140b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:55.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.977+0000 7f3113fff640 1 -- 172.21.15.135:0/814317887 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311415f5c0 con 0x7f3114158a50 2024-04-17T06:09:55.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.977+0000 7f3112ffd640 1 -- 172.21.15.135:0/814317887 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1506912727 0 0) 0x7f311415fb80 con 0x7f3114158e30 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f3113fff640 1 --2- 172.21.15.135:0/814317887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f311415ec20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f310400aa10 tx=0x7f310402cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e521a306f2297231 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f3112ffd640 1 -- 172.21.15.135:0/814317887 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310403a070 con 0x7f3114158a50 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f3112ffd640 1 -- 172.21.15.135:0/814317887 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310402dca0 con 0x7f3114158a50 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f3112ffd640 1 -- 172.21.15.135:0/814317887 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310400b040 con 0x7f3114158a50 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f311a6eb640 1 -- 172.21.15.135:0/814317887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 msgr2=0x7f311415ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f311a6eb640 1 --2- 172.21.15.135:0/814317887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f311415ec20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f310400aa10 tx=0x7f310402cef0 comp rx=0 tx=0).stop 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f311a6eb640 1 -- 172.21.15.135:0/814317887 shutdown_connections 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f311a6eb640 1 --2- 172.21.15.135:0/814317887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f311415ec20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f311a6eb640 1 -- 172.21.15.135:0/814317887 >> 172.21.15.135:0/814317887 conn(0x7f31140d0520 msgr2=0x7f31140d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:55.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f311a6eb640 1 -- 172.21.15.135:0/814317887 shutdown_connections 2024-04-17T06:09:55.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.978+0000 7f311a6eb640 1 -- 172.21.15.135:0/814317887 wait complete. 2024-04-17T06:09:55.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.979+0000 7f311a6eb640 1 Processor -- start 2024-04-17T06:09:55.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.980+0000 7f311a6eb640 1 -- start start 2024-04-17T06:09:55.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.980+0000 7f311a6eb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f311406f610 0x7f31140ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.980+0000 7f311a6eb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f31140c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.980+0000 7f311a6eb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3114158e30 0x7f311415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.980+0000 7f311a6eb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31140b3ca0 con 0x7f3114158a50 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.980+0000 7f311a6eb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f311406faa0 con 0x7f311406f610 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.980+0000 7f311a6eb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f31141497f0 con 0x7f3114158e30 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f31140c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f31140c5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54832/0 (socket says 172.21.15.135:54832) 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 -- 172.21.15.135:0/1760422141 learned_addr learned my addr 172.21.15.135:0/1760422141 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f3113fff640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f311406f610 0x7f31140ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f3118c88640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3114158e30 0x7f311415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 -- 172.21.15.135:0/1760422141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3114158e30 msgr2=0x7f311415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3114158e30 0x7f311415e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:55.981 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 -- 172.21.15.135:0/1760422141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f311406f610 msgr2=0x7f31140ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f311406f610 0x7f31140ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 -- 172.21.15.135:0/1760422141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3114149a40 con 0x7f3114158a50 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f31137fe640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f31140c5500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f310800c3f0 tx=0x7f310800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f3110ff9640 1 -- 172.21.15.135:0/1760422141 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080074e0 con 0x7f3114158a50 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f3110ff9640 1 -- 172.21.15.135:0/1760422141 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31080047e0 con 0x7f3114158a50 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f3110ff9640 1 -- 172.21.15.135:0/1760422141 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108004c10 con 0x7f3114158a50 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3114149d20 con 0x7f3114158a50 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.981+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311414a2e0 con 0x7f3114158a50 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.982+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f311414a4e0 con 0x7f3114158a50 2024-04-17T06:09:55.982 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.983+0000 7f3110ff9640 1 -- 172.21.15.135:0/1760422141 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3108011070 con 0x7f3114158a50 2024-04-17T06:09:55.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.983+0000 7f3110ff9640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f310006c8f0 0x7f310006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:55.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.983+0000 7f3110ff9640 1 -- 172.21.15.135:0/1760422141 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f310808b8f0 con 0x7f3114158a50 2024-04-17T06:09:55.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.983+0000 7f3113fff640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f310006c8f0 0x7f310006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:55.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.984+0000 7f3113fff640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f310006c8f0 0x7f310006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3104037000 tx=0x7f3104008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:55.985 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:55.988+0000 7f3110ff9640 1 -- 172.21.15.135:0/1760422141 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f311414a4e0 con 0x7f3114158a50 2024-04-17T06:09:56.160 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.162+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3114074940 con 0x7f3114158a50 2024-04-17T06:09:56.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.164+0000 7f3110ff9640 1 -- 172.21.15.135:0/1760422141 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f3108059420 con 0x7f3114158a50 2024-04-17T06:09:56.162 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:56.162 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:56.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f310006c8f0 msgr2=0x7f310006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f310006c8f0 0x7f310006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3104037000 tx=0x7f3104008040 comp rx=0 tx=0).stop 2024-04-17T06:09:56.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 msgr2=0x7f31140c5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f31140c5500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f310800c3f0 tx=0x7f310800c8b0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 shutdown_connections 2024-04-17T06:09:56.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3114158e30 0x7f311415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f310006c8f0 0x7f310006edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.164 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3114158a50 0x7f31140c5500 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 --2- 172.21.15.135:0/1760422141 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f311406f610 0x7f31140ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 >> 172.21.15.135:0/1760422141 conn(0x7f31140d0520 msgr2=0x7f31141570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:56.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 shutdown_connections 2024-04-17T06:09:56.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.166+0000 7f311a6eb640 1 -- 172.21.15.135:0/1760422141 wait complete. 2024-04-17T06:09:56.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:09:56.176 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:56.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.283+0000 7fb8dc996640 1 Processor -- start 2024-04-17T06:09:56.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.283+0000 7fb8dc996640 1 -- start start 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8dc996640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb8d80b3590 0x7fb8d80af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8dc996640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8d80afc00 0x7fb8d80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8dc996640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d80b3a50 con 0x7fb8d815bb00 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8dc996640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d80b5860 con 0x7fb8d80afc00 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8dc996640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d80b4660 con 0x7fb8d80b3590 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8d6d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8d80afc00 0x7fb8d80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8d6d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8d80afc00 0x7fb8d80ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54836/0 (socket says 172.21.15.135:54836) 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8d6d9d640 1 -- 172.21.15.135:0/1950986109 learned_addr learned my addr 172.21.15.135:0/1950986109 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8d5d9b640 1 --2- 172.21.15.135:0/1950986109 >> v2:172.21.15.190:3300/0 conn(0x7fb8d80b3590 0x7fb8d80af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8d6d9d640 1 -- 172.21.15.135:0/1950986109 >> v2:172.21.15.190:3300/0 conn(0x7fb8d80b3590 msgr2=0x7fb8d80af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:56.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8d6d9d640 1 --2- 172.21.15.135:0/1950986109 >> v2:172.21.15.190:3300/0 conn(0x7fb8d80b3590 0x7fb8d80af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8d6d9d640 1 -- 172.21.15.135:0/1950986109 >> v1:172.21.15.190:6789/0 conn(0x7fb8d815bb00 legacy=0x7fb8d80b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.284+0000 7fb8d6d9d640 1 -- 172.21.15.135:0/1950986109 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d80b3d30 con 0x7fb8d80afc00 2024-04-17T06:09:56.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.285+0000 7fb8d6d9d640 1 --2- 172.21.15.135:0/1950986109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d80afc00 0x7fb8d80ce720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb8d806dd80 tx=0x7fb8c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ee36e130f45d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.285+0000 7fb8d559a640 1 -- 172.21.15.135:0/1950986109 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8034070 con 0x7fb8d80afc00 2024-04-17T06:09:56.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.285+0000 7fb8d559a640 1 -- 172.21.15.135:0/1950986109 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c8030860 con 0x7fb8d80afc00 2024-04-17T06:09:56.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.285+0000 7fb8d559a640 1 -- 172.21.15.135:0/1950986109 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8030c30 con 0x7fb8d80afc00 2024-04-17T06:09:56.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.286+0000 7fb8dc996640 1 -- 172.21.15.135:0/1950986109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d80afc00 msgr2=0x7fb8d80ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.285 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.286+0000 7fb8dc996640 1 --2- 172.21.15.135:0/1950986109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d80afc00 0x7fb8d80ce720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb8d806dd80 tx=0x7fb8c802b9a0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.286+0000 7fb8dc996640 1 -- 172.21.15.135:0/1950986109 shutdown_connections 2024-04-17T06:09:56.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.286+0000 7fb8dc996640 1 --2- 172.21.15.135:0/1950986109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d80afc00 0x7fb8d80ce720 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.286+0000 7fb8dc996640 1 --2- 172.21.15.135:0/1950986109 >> v2:172.21.15.190:3300/0 conn(0x7fb8d80b3590 0x7fb8d80af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.286+0000 7fb8dc996640 1 -- 172.21.15.135:0/1950986109 >> 172.21.15.135:0/1950986109 conn(0x7fb8d80d0520 msgr2=0x7fb8d80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:56.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.286+0000 7fb8dc996640 1 -- 172.21.15.135:0/1950986109 shutdown_connections 2024-04-17T06:09:56.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.286+0000 7fb8dc996640 1 -- 172.21.15.135:0/1950986109 wait complete. 2024-04-17T06:09:56.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.287+0000 7fb8dc996640 1 Processor -- start 2024-04-17T06:09:56.286 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.287+0000 7fb8dc996640 1 -- start start 2024-04-17T06:09:56.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8dc996640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d80afc00 0x7fb8d8149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8dc996640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8d80b3590 0x7fb8d815bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8dc996640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d815bb00 0x7fb8d80723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8dc996640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d80cec60 con 0x7fb8d815bb00 2024-04-17T06:09:56.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8dc996640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d80b42f0 con 0x7fb8d80afc00 2024-04-17T06:09:56.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8dc996640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d80cede0 con 0x7fb8d80b3590 2024-04-17T06:09:56.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d6d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d815bb00 0x7fb8d80723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.289 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d6d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d815bb00 0x7fb8d80723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54846/0 (socket says 172.21.15.135:54846) 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d6d9d640 1 -- 172.21.15.135:0/2099742031 learned_addr learned my addr 172.21.15.135:0/2099742031 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d659c640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d80afc00 0x7fb8d8149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d5d9b640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8d80b3590 0x7fb8d815bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d6d9d640 1 -- 172.21.15.135:0/2099742031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8d80b3590 msgr2=0x7fb8d815bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d6d9d640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8d80b3590 0x7fb8d815bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d6d9d640 1 -- 172.21.15.135:0/2099742031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d80afc00 msgr2=0x7fb8d8149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d6d9d640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d80afc00 0x7fb8d8149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.288+0000 7fb8d6d9d640 1 -- 172.21.15.135:0/2099742031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d815da90 con 0x7fb8d815bb00 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.289+0000 7fb8d6d9d640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d815bb00 0x7fb8d80723e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb8c802b9e0 tx=0x7fb8c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.289+0000 7fb8c6ffd640 1 -- 172.21.15.135:0/2099742031 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8048070 con 0x7fb8d815bb00 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.289+0000 7fb8c6ffd640 1 -- 172.21.15.135:0/2099742031 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c8030660 con 0x7fb8d815bb00 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.289+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d815dd30 con 0x7fb8d815bb00 2024-04-17T06:09:56.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.289+0000 7fb8c6ffd640 1 -- 172.21.15.135:0/2099742031 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8034040 con 0x7fb8d815bb00 2024-04-17T06:09:56.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.289+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d815e2f0 con 0x7fb8d815bb00 2024-04-17T06:09:56.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.290+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d8074940 con 0x7fb8d815bb00 2024-04-17T06:09:56.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.290+0000 7fb8c6ffd640 1 -- 172.21.15.135:0/2099742031 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8c804b070 con 0x7fb8d815bb00 2024-04-17T06:09:56.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.291+0000 7fb8c6ffd640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8b806c8f0 0x7fb8b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.291+0000 7fb8c6ffd640 1 -- 172.21.15.135:0/2099742031 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb8c80add80 con 0x7fb8d815bb00 2024-04-17T06:09:56.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.291+0000 7fb8d659c640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8b806c8f0 0x7fb8b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.291+0000 7fb8d659c640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8b806c8f0 0x7fb8b806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0001e70 tx=0x7fb8d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.295+0000 7fb8c6ffd640 1 -- 172.21.15.135:0/2099742031 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8c807b9d0 con 0x7fb8d815bb00 2024-04-17T06:09:56.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.469+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8d806ce80 con 0x7fb8d815bb00 2024-04-17T06:09:56.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.469+0000 7fb8c6ffd640 1 -- 172.21.15.135:0/2099742031 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fb8c807fcd0 con 0x7fb8d815bb00 2024-04-17T06:09:56.468 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8b806c8f0 msgr2=0x7fb8b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8b806c8f0 0x7fb8b806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0001e70 tx=0x7fb8d0009040 comp rx=0 tx=0).stop 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d815bb00 msgr2=0x7fb8d80723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d815bb00 0x7fb8d80723e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb8c802b9e0 tx=0x7fb8c8004730 comp rx=0 tx=0).stop 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 shutdown_connections 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8b806c8f0 0x7fb8b806edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8d815bb00 0x7fb8d80723e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8d80b3590 0x7fb8d815bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 --2- 172.21.15.135:0/2099742031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8d80afc00 0x7fb8d8149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 >> 172.21.15.135:0/2099742031 conn(0x7fb8d80d0520 msgr2=0x7fb8d8157720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:56.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 shutdown_connections 2024-04-17T06:09:56.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.471+0000 7fb8dc996640 1 -- 172.21.15.135:0/2099742031 wait complete. 2024-04-17T06:09:56.481 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:09:56.586 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.587+0000 7ff1ab463640 1 Processor -- start 2024-04-17T06:09:56.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.588+0000 7ff1ab463640 1 -- start start 2024-04-17T06:09:56.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.588+0000 7ff1ab463640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cf690 0x7ff1a40ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.588+0000 7ff1ab463640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1a40cc470 0x7ff1a40ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.588+0000 7ff1ab463640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a415eff0 con 0x7ff1a415ec10 2024-04-17T06:09:56.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.588+0000 7ff1ab463640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a40ad860 con 0x7ff1a40cc470 2024-04-17T06:09:56.587 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.588+0000 7ff1ab463640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a40cc970 con 0x7ff1a40cf690 2024-04-17T06:09:56.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a89fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cf690 0x7ff1a40ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a9a00640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1a40cc470 0x7ff1a40ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a89fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cf690 0x7ff1a40ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:36540/0 (socket says 172.21.15.135:36540) 2024-04-17T06:09:56.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a89fe640 1 -- 172.21.15.135:0/4123585404 learned_addr learned my addr 172.21.15.135:0/4123585404 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:56.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a91ff640 1 --1- 172.21.15.135:0/4123585404 >> v1:172.21.15.190:6789/0 conn(0x7ff1a415ec10 0x7ff1a40cef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:46726/0 (socket says 172.21.15.135:46726) 2024-04-17T06:09:56.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a9a00640 1 -- 172.21.15.135:0/4123585404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cf690 msgr2=0x7ff1a40ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:56.589 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a9a00640 1 --2- 172.21.15.135:0/4123585404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cf690 0x7ff1a40ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a9a00640 1 -- 172.21.15.135:0/4123585404 >> v1:172.21.15.190:6789/0 conn(0x7ff1a415ec10 legacy=0x7ff1a40cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a9a00640 1 -- 172.21.15.135:0/4123585404 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a40ccc50 con 0x7ff1a40cc470 2024-04-17T06:09:56.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff19bfff640 1 -- 172.21.15.135:0/4123585404 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2486422105 0 0) 0x7ff1a415eff0 con 0x7ff1a415ec10 2024-04-17T06:09:56.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.589+0000 7ff1a89fe640 1 --2- 172.21.15.135:0/4123585404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cf690 0x7ff1a40ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:09:56.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.590+0000 7ff1a9a00640 1 --2- 172.21.15.135:0/4123585404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a40cc470 0x7ff1a40ae7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009d90 tx=0x7ff19c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2404a888cbcbcb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.590+0000 7ff19bfff640 1 -- 172.21.15.135:0/4123585404 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c039070 con 0x7ff1a40cc470 2024-04-17T06:09:56.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.590+0000 7ff19bfff640 1 -- 172.21.15.135:0/4123585404 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c02cbc0 con 0x7ff1a40cc470 2024-04-17T06:09:56.590 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.590+0000 7ff19bfff640 1 -- 172.21.15.135:0/4123585404 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c034440 con 0x7ff1a40cc470 2024-04-17T06:09:56.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.591+0000 7ff1ab463640 1 -- 172.21.15.135:0/4123585404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a40cc470 msgr2=0x7ff1a40ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.591+0000 7ff1ab463640 1 --2- 172.21.15.135:0/4123585404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a40cc470 0x7ff1a40ae7f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009d90 tx=0x7ff19c02be10 comp rx=0 tx=0).stop 2024-04-17T06:09:56.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.591+0000 7ff1ab463640 1 -- 172.21.15.135:0/4123585404 shutdown_connections 2024-04-17T06:09:56.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.591+0000 7ff1ab463640 1 --2- 172.21.15.135:0/4123585404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a40cc470 0x7ff1a40ae7f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.591 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.591+0000 7ff1ab463640 1 --2- 172.21.15.135:0/4123585404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cf690 0x7ff1a40ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.591+0000 7ff1ab463640 1 -- 172.21.15.135:0/4123585404 >> 172.21.15.135:0/4123585404 conn(0x7ff1a40d0930 msgr2=0x7ff1a40d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:56.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.591+0000 7ff1ab463640 1 -- 172.21.15.135:0/4123585404 shutdown_connections 2024-04-17T06:09:56.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.591+0000 7ff1ab463640 1 -- 172.21.15.135:0/4123585404 wait complete. 2024-04-17T06:09:56.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.592+0000 7ff1ab463640 1 Processor -- start 2024-04-17T06:09:56.592 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.592+0000 7ff1ab463640 1 -- start start 2024-04-17T06:09:56.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1ab463640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cc470 0x7ff1a41f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1ab463640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a40cf690 0x7ff1a4149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1ab463640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a415ec10 0x7ff1a414a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1ab463640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a40ce6c0 con 0x7ff1a40cf690 2024-04-17T06:09:56.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1ab463640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a40add90 con 0x7ff1a415ec10 2024-04-17T06:09:56.594 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1ab463640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a40ce840 con 0x7ff1a40cc470 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a40cf690 0x7ff1a4149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a40cf690 0x7ff1a4149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54848/0 (socket says 172.21.15.135:54848) 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 -- 172.21.15.135:0/1548104626 learned_addr learned my addr 172.21.15.135:0/1548104626 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a91ff640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cc470 0x7ff1a41f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a9a00640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a415ec10 0x7ff1a414a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 -- 172.21.15.135:0/1548104626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cc470 msgr2=0x7ff1a41f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cc470 0x7ff1a41f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 -- 172.21.15.135:0/1548104626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a415ec10 msgr2=0x7ff1a414a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a415ec10 0x7ff1a414a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 -- 172.21.15.135:0/1548104626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a414a7a0 con 0x7ff1a40cf690 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.593+0000 7ff1a89fe640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a40cf690 0x7ff1a4149b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff19400e9d0 tx=0x7ff19400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.594+0000 7ff199ffb640 1 -- 172.21.15.135:0/1548104626 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194017070 con 0x7ff1a40cf690 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.594+0000 7ff199ffb640 1 -- 172.21.15.135:0/1548104626 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19400ca60 con 0x7ff1a40cf690 2024-04-17T06:09:56.595 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.594+0000 7ff199ffb640 1 -- 172.21.15.135:0/1548104626 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19400ce10 con 0x7ff1a40cf690 2024-04-17T06:09:56.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.594+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a40b6320 con 0x7ff1a40cf690 2024-04-17T06:09:56.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.594+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a406e5b0 con 0x7ff1a40cf690 2024-04-17T06:09:56.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.595+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a406e3a0 con 0x7ff1a40cf690 2024-04-17T06:09:56.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.595+0000 7ff199ffb640 1 -- 172.21.15.135:0/1548104626 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff19401a070 con 0x7ff1a40cf690 2024-04-17T06:09:56.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.596+0000 7ff199ffb640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff18806c8f0 0x7ff18806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.596+0000 7ff1a91ff640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff18806c8f0 0x7ff18806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.596+0000 7ff199ffb640 1 -- 172.21.15.135:0/1548104626 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff19408bcd0 con 0x7ff1a40cf690 2024-04-17T06:09:56.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.596+0000 7ff1a91ff640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff18806c8f0 0x7ff18806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0007fd0 tx=0x7ff1a0007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.600+0000 7ff199ffb640 1 -- 172.21.15.135:0/1548104626 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1a414a7a0 con 0x7ff1a40cf690 2024-04-17T06:09:56.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.778+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff1a4074940 con 0x7ff1a40cf690 2024-04-17T06:09:56.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.778+0000 7ff199ffb640 1 -- 172.21.15.135:0/1548104626 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7ff194059800 con 0x7ff1a40cf690 2024-04-17T06:09:56.777 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:56.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff18806c8f0 msgr2=0x7ff18806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff18806c8f0 0x7ff18806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0007fd0 tx=0x7ff1a0007de0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a40cf690 msgr2=0x7ff1a4149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a40cf690 0x7ff1a4149b00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff19400e9d0 tx=0x7ff19400ee90 comp rx=0 tx=0).stop 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 shutdown_connections 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1a415ec10 0x7ff1a414a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff18806c8f0 0x7ff18806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1a40cf690 0x7ff1a4149b00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 --2- 172.21.15.135:0/1548104626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1a40cc470 0x7ff1a41f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 >> 172.21.15.135:0/1548104626 conn(0x7ff1a40d0930 msgr2=0x7ff1a40cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 shutdown_connections 2024-04-17T06:09:56.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.780+0000 7ff1ab463640 1 -- 172.21.15.135:0/1548104626 wait complete. 2024-04-17T06:09:56.791 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:09:56.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.897+0000 7f9c8071c640 1 Processor -- start 2024-04-17T06:09:56.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.897+0000 7f9c8071c640 1 -- start start 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c8071c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9c7809a9f0 0x7f9c7809add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c8071c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c7809b310 con 0x7f9c780c78d0 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c8071c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c780d0c90 con 0x7f9c7815cc60 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c8071c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9c780d0f70 con 0x7f9c7809a9f0 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7ecb9640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9c780c78d0 0x7f9c780cda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51632/0 (socket says 172.21.15.135:51632) 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7ecb9640 1 -- 172.21.15.135:0/985173592 learned_addr learned my addr 172.21.15.135:0/985173592 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7e4b8640 1 --2- 172.21.15.135:0/985173592 >> v2:172.21.15.190:3300/0 conn(0x7f9c7809a9f0 0x7f9c7809add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3651957480 0 0) 0x7f9c7809b310 con 0x7f9c780c78d0 2024-04-17T06:09:56.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c64003520 con 0x7f9c780c78d0 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2316215247 0 0) 0x7f9c64003520 con 0x7f9c780c78d0 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 >> v2:172.21.15.190:3300/0 conn(0x7f9c7809a9f0 msgr2=0x7f9c7809add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7d4b6640 1 --2- 172.21.15.135:0/985173592 >> v2:172.21.15.190:3300/0 conn(0x7f9c7809a9f0 0x7f9c7809add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.898+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 >> v1:172.21.15.190:6789/0 conn(0x7f9c7815cc60 legacy=0x7f9c780cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c780d1250 con 0x7f9c780c78d0 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f9c700019f0 con 0x7f9c780c78d0 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 >> v1:172.21.15.135:6789/0 conn(0x7f9c780c78d0 legacy=0x7f9c780cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c640039b0 0x7f9c64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c640043b0 0x7f9c64004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c64005080 0x7f9c6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780d0f70 con 0x7f9c640039b0 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6400bfe0 con 0x7f9c64005080 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6400c1e0 con 0x7f9c640043b0 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7e4b8640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c640039b0 0x7f9c64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7ecb9640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c64005080 0x7f9c6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7dcb7640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c640043b0 0x7f9c64004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7e4b8640 1 -- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c640043b0 msgr2=0x7f9c64004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7e4b8640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c640043b0 0x7f9c64004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7e4b8640 1 -- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c64005080 msgr2=0x7f9c6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7e4b8640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c64005080 0x7f9c6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7e4b8640 1 -- 172.21.15.135:0/985173592 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c64003520 con 0x7f9c640039b0 2024-04-17T06:09:56.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7ecb9640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c64005080 0x7f9c6400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.899+0000 7f9c7ecb9640 1 -- 172.21.15.135:0/985173592 reap_dead start 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c7e4b8640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c640039b0 0x7f9c64003d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009ac0 tx=0x7f9c64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da36c1ab1907774 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74039070 con 0x7f9c640039b0 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7402c6b0 con 0x7f9c640039b0 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c7d4b6640 1 -- 172.21.15.135:0/985173592 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74009690 con 0x7f9c640039b0 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c8071c640 1 -- 172.21.15.135:0/985173592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c640039b0 msgr2=0x7f9c64003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c8071c640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c640039b0 0x7f9c64003d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009ac0 tx=0x7f9c64005830 comp rx=0 tx=0).stop 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c8071c640 1 -- 172.21.15.135:0/985173592 shutdown_connections 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c8071c640 1 --2- 172.21.15.135:0/985173592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c640039b0 0x7f9c64003d90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c8071c640 1 -- 172.21.15.135:0/985173592 >> 172.21.15.135:0/985173592 conn(0x7f9c780b41a0 msgr2=0x7f9c780b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.900+0000 7f9c8071c640 1 -- 172.21.15.135:0/985173592 shutdown_connections 2024-04-17T06:09:56.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.901+0000 7f9c8071c640 1 -- 172.21.15.135:0/985173592 wait complete. 2024-04-17T06:09:56.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c8071c640 1 Processor -- start 2024-04-17T06:09:56.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c8071c640 1 -- start start 2024-04-17T06:09:56.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c8071c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c640039b0 0x7f9c780b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c8071c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c781486f0 0x7f9c780b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c8071c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c7814b760 0x7f9c780cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c8071c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780cca60 con 0x7f9c781486f0 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c8071c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9c780c63a0 con 0x7f9c640039b0 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c8071c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7809bdf0 con 0x7f9c7814b760 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c7dcb7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c781486f0 0x7f9c780b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.902+0000 7f9c7dcb7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c781486f0 0x7f9c780b7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54870/0 (socket says 172.21.15.135:54870) 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c7dcb7640 1 -- 172.21.15.135:0/4010568048 learned_addr learned my addr 172.21.15.135:0/4010568048 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c7e4b8640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c640039b0 0x7f9c780b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c7dcb7640 1 -- 172.21.15.135:0/4010568048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c7814b760 msgr2=0x7f9c780cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c7dcb7640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c7814b760 0x7f9c780cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c7dcb7640 1 -- 172.21.15.135:0/4010568048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c640039b0 msgr2=0x7f9c780b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c7dcb7640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c640039b0 0x7f9c780b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c7dcb7640 1 -- 172.21.15.135:0/4010568048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c780b8230 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c7dcb7640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c781486f0 0x7f9c780b7cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e750 tx=0x7f9c6800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c6effd640 1 -- 172.21.15.135:0/4010568048 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c680050d0 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c6effd640 1 -- 172.21.15.135:0/4010568048 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c68018070 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c6effd640 1 -- 172.21.15.135:0/4010568048 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68005490 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c780b6c90 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.903+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c780b7210 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.904+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c78074940 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.905+0000 7f9c6effd640 1 -- 172.21.15.135:0/4010568048 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c6800c6c0 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.905+0000 7f9c6effd640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9c5c06c8f0 0x7f9c5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.905+0000 7f9c6effd640 1 -- 172.21.15.135:0/4010568048 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9c6808bba0 con 0x7f9c781486f0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.905+0000 7f9c7e4b8640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9c5c06c8f0 0x7f9c5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:56.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.906+0000 7f9c7e4b8640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9c5c06c8f0 0x7f9c5c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c74036000 tx=0x7f9c74030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:56.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:56.909+0000 7f9c6effd640 1 -- 172.21.15.135:0/4010568048 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c68059c90 con 0x7f9c781486f0 2024-04-17T06:09:57.086 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.087+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9c7809b610 con 0x7f9c781486f0 2024-04-17T06:09:57.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.087+0000 7f9c6effd640 1 -- 172.21.15.135:0/4010568048 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f9c6808b070 con 0x7f9c781486f0 2024-04-17T06:09:57.087 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9c5c06c8f0 msgr2=0x7f9c5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9c5c06c8f0 0x7f9c5c06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c74036000 tx=0x7f9c74030000 comp rx=0 tx=0).stop 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c781486f0 msgr2=0x7f9c780b7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c781486f0 0x7f9c780b7cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e750 tx=0x7f9c6800ec10 comp rx=0 tx=0).stop 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 shutdown_connections 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9c7814b760 0x7f9c780cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9c5c06c8f0 0x7f9c5c06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c781486f0 0x7f9c780b7cf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 --2- 172.21.15.135:0/4010568048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9c640039b0 0x7f9c780b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.089+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 >> 172.21.15.135:0/4010568048 conn(0x7f9c780b41a0 msgr2=0x7f9c78074e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:57.089 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.090+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 shutdown_connections 2024-04-17T06:09:57.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.090+0000 7f9c8071c640 1 -- 172.21.15.135:0/4010568048 wait complete. 2024-04-17T06:09:57.099 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:09:57.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.205+0000 7f1cf7474640 1 Processor -- start 2024-04-17T06:09:57.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.205+0000 7f1cf7474640 1 -- start start 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf7474640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf015ff30 0x7f1cf015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf7474640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1cf0071ec0 0x7f1cf0072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf7474640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1cf00cf740 0x7f1cf00cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf7474640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf006df20 con 0x7f1cf0071ec0 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf7474640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf006eaa0 con 0x7f1cf00cf740 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf7474640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf006e170 con 0x7f1cf015ff30 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf4a0f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1cf0071ec0 0x7f1cf0072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf4a0f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1cf0071ec0 0x7f1cf0072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54874/0 (socket says 172.21.15.135:54874) 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf4a0f640 1 -- 172.21.15.135:0/3796117987 learned_addr learned my addr 172.21.15.135:0/3796117987 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf5210640 1 --2- 172.21.15.135:0/3796117987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf015ff30 0x7f1cf015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf5a11640 1 --2- 172.21.15.135:0/3796117987 >> v2:172.21.15.190:3300/0 conn(0x7f1cf00cf740 0x7f1cf00cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf4a0f640 1 -- 172.21.15.135:0/3796117987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf015ff30 msgr2=0x7f1cf015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:57.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.206+0000 7f1cf4a0f640 1 --2- 172.21.15.135:0/3796117987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf015ff30 0x7f1cf015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.207+0000 7f1cf4a0f640 1 -- 172.21.15.135:0/3796117987 >> v2:172.21.15.190:3300/0 conn(0x7f1cf00cf740 msgr2=0x7f1cf00cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:57.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.207+0000 7f1cf4a0f640 1 --2- 172.21.15.135:0/3796117987 >> v2:172.21.15.190:3300/0 conn(0x7f1cf00cf740 0x7f1cf00cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.207+0000 7f1cf4a0f640 1 -- 172.21.15.135:0/3796117987 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf006e450 con 0x7f1cf0071ec0 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.207+0000 7f1cf4a0f640 1 --2- 172.21.15.135:0/3796117987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf0072340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00741f0 tx=0x7f1ce402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a045fa24257ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.207+0000 7f1ceffff640 1 -- 172.21.15.135:0/3796117987 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4035020 con 0x7f1cf0071ec0 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.207+0000 7f1ceffff640 1 -- 172.21.15.135:0/3796117987 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce4031870 con 0x7f1cf0071ec0 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.207+0000 7f1ceffff640 1 -- 172.21.15.135:0/3796117987 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4031c40 con 0x7f1cf0071ec0 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 -- 172.21.15.135:0/3796117987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 msgr2=0x7f1cf0072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 --2- 172.21.15.135:0/3796117987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf0072340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00741f0 tx=0x7f1ce402b910 comp rx=0 tx=0).stop 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 -- 172.21.15.135:0/3796117987 shutdown_connections 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 --2- 172.21.15.135:0/3796117987 >> v2:172.21.15.190:3300/0 conn(0x7f1cf00cf740 0x7f1cf00cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 --2- 172.21.15.135:0/3796117987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf0072340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 --2- 172.21.15.135:0/3796117987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf015ff30 0x7f1cf015cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 -- 172.21.15.135:0/3796117987 >> 172.21.15.135:0/3796117987 conn(0x7f1cf00d0520 msgr2=0x7f1cf00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:57.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 -- 172.21.15.135:0/3796117987 shutdown_connections 2024-04-17T06:09:57.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.208+0000 7f1cf7474640 1 -- 172.21.15.135:0/3796117987 wait complete. 2024-04-17T06:09:57.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.209+0000 7f1cf7474640 1 Processor -- start 2024-04-17T06:09:57.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.209+0000 7f1cf7474640 1 -- start start 2024-04-17T06:09:57.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf7474640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf015c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf7474640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf00cf740 0x7f1cf00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf7474640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf00cbf60 0x7f1cf00b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf7474640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00d0290 con 0x7f1cf0071ec0 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf7474640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00d0110 con 0x7f1cf00cf740 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf7474640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf0160390 con 0x7f1cf00cbf60 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54888/0 (socket says 172.21.15.135:54888) 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 -- 172.21.15.135:0/2317180922 learned_addr learned my addr 172.21.15.135:0/2317180922 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 -- 172.21.15.135:0/2317180922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf00cbf60 msgr2=0x7f1cf00b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf00cbf60 0x7f1cf00b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.213 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 -- 172.21.15.135:0/2317180922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf00cf740 msgr2=0x7f1cf00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf00cf740 0x7f1cf00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 -- 172.21.15.135:0/2317180922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf0139930 con 0x7f1cf0071ec0 2024-04-17T06:09:57.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.210+0000 7f1cf5210640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf015c590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0007e70 tx=0x7f1ce000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.211+0000 7f1cedffb640 1 -- 172.21.15.135:0/2317180922 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0014620 con 0x7f1cf0071ec0 2024-04-17T06:09:57.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.211+0000 7f1cedffb640 1 -- 172.21.15.135:0/2317180922 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce0019070 con 0x7f1cf0071ec0 2024-04-17T06:09:57.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.211+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf0139c10 con 0x7f1cf0071ec0 2024-04-17T06:09:57.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.211+0000 7f1cedffb640 1 -- 172.21.15.135:0/2317180922 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0014a00 con 0x7f1cf0071ec0 2024-04-17T06:09:57.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.211+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf00b0a20 con 0x7f1cf0071ec0 2024-04-17T06:09:57.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.212+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf0074940 con 0x7f1cf0071ec0 2024-04-17T06:09:57.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.212+0000 7f1cedffb640 1 -- 172.21.15.135:0/2317180922 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ce0014ba0 con 0x7f1cf0071ec0 2024-04-17T06:09:57.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.212+0000 7f1cedffb640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.213+0000 7f1cedffb640 1 -- 172.21.15.135:0/2317180922 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1ce008c6a0 con 0x7f1cf0071ec0 2024-04-17T06:09:57.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.213+0000 7f1cf4a0f640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.213+0000 7f1cf4a0f640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4036000 tx=0x7f1ce4030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.216 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.217+0000 7f1cedffb640 1 -- 172.21.15.135:0/2317180922 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ce005a820 con 0x7f1cf0071ec0 2024-04-17T06:09:57.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.393+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1cf00729f0 con 0x7f1cf0071ec0 2024-04-17T06:09:57.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.393+0000 7f1cedffb640 1 -- 172.21.15.135:0/2317180922 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f1ce005a120 con 0x7f1cf0071ec0 2024-04-17T06:09:57.393 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1cdc06c8f0 msgr2=0x7f1cdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4036000 tx=0x7f1ce4030000 comp rx=0 tx=0).stop 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 msgr2=0x7f1cf015c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf015c590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0007e70 tx=0x7f1ce000e4b0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 shutdown_connections 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1cf00cbf60 0x7f1cf00b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1cf00cf740 0x7f1cf00cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 --2- 172.21.15.135:0/2317180922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1cf0071ec0 0x7f1cf015c590 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.395+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 >> 172.21.15.135:0/2317180922 conn(0x7f1cf00d0520 msgr2=0x7f1cf015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:57.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.396+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 shutdown_connections 2024-04-17T06:09:57.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.396+0000 7f1cf7474640 1 -- 172.21.15.135:0/2317180922 wait complete. 2024-04-17T06:09:57.406 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:57.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.510+0000 7f49e2fe2640 1 Processor -- start 2024-04-17T06:09:57.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.510+0000 7f49e2fe2640 1 -- start start 2024-04-17T06:09:57.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e2fe2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f49dc0b3590 0x7f49dc0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e2fe2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49dc0afc00 0x7f49dc0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e2fe2640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49dc0b3a50 con 0x7f49dc15bb00 2024-04-17T06:09:57.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e2fe2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f49dc0b5860 con 0x7f49dc0afc00 2024-04-17T06:09:57.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e2fe2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f49dc0b4660 con 0x7f49dc0b3590 2024-04-17T06:09:57.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e157f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49dc0afc00 0x7f49dc0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e0d7e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f49dc15bb00 0x7f49dc0b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51646/0 (socket says 172.21.15.135:51646) 2024-04-17T06:09:57.511 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e157f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49dc0afc00 0x7f49dc0ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54898/0 (socket says 172.21.15.135:54898) 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e0d7e640 1 -- 172.21.15.135:0/2850251761 learned_addr learned my addr 172.21.15.135:0/2850251761 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49d3fff640 1 --2- 172.21.15.135:0/2850251761 >> v2:172.21.15.190:3300/0 conn(0x7f49dc0b3590 0x7f49dc0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e157f640 1 -- 172.21.15.135:0/2850251761 >> v2:172.21.15.190:3300/0 conn(0x7f49dc0b3590 msgr2=0x7f49dc0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e157f640 1 --2- 172.21.15.135:0/2850251761 >> v2:172.21.15.190:3300/0 conn(0x7f49dc0b3590 0x7f49dc0af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e157f640 1 -- 172.21.15.135:0/2850251761 >> v1:172.21.15.135:6789/0 conn(0x7f49dc15bb00 legacy=0x7f49dc0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49e157f640 1 -- 172.21.15.135:0/2850251761 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc0b3d30 con 0x7f49dc0afc00 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49d37fe640 1 -- 172.21.15.135:0/2850251761 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1802778525 0 0) 0x7f49dc0b3a50 con 0x7f49dc15bb00 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.511+0000 7f49d3fff640 1 --2- 172.21.15.135:0/2850251761 >> v2:172.21.15.190:3300/0 conn(0x7f49dc0b3590 0x7f49dc0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.512+0000 7f49e157f640 1 --2- 172.21.15.135:0/2850251761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc0ce720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f49dc06dd80 tx=0x7f49d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2cb8d95186218b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.512+0000 7f49d37fe640 1 -- 172.21.15.135:0/2850251761 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4039070 con 0x7f49dc0afc00 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.512+0000 7f49d37fe640 1 -- 172.21.15.135:0/2850251761 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d402cc80 con 0x7f49dc0afc00 2024-04-17T06:09:57.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.512+0000 7f49d37fe640 1 -- 172.21.15.135:0/2850251761 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4034660 con 0x7f49dc0afc00 2024-04-17T06:09:57.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.513+0000 7f49e2fe2640 1 -- 172.21.15.135:0/2850251761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 msgr2=0x7f49dc0ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.513+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/2850251761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc0ce720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f49dc06dd80 tx=0x7f49d402bed0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.513+0000 7f49e2fe2640 1 -- 172.21.15.135:0/2850251761 shutdown_connections 2024-04-17T06:09:57.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.513+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/2850251761 >> v2:172.21.15.190:3300/0 conn(0x7f49dc0b3590 0x7f49dc0af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.513+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/2850251761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc0ce720 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.513+0000 7f49e2fe2640 1 -- 172.21.15.135:0/2850251761 >> 172.21.15.135:0/2850251761 conn(0x7f49dc0d0520 msgr2=0x7f49dc0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.513+0000 7f49e2fe2640 1 -- 172.21.15.135:0/2850251761 shutdown_connections 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.513+0000 7f49e2fe2640 1 -- 172.21.15.135:0/2850251761 wait complete. 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.514+0000 7f49e2fe2640 1 Processor -- start 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.514+0000 7f49e2fe2640 1 -- start start 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e2fe2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e2fe2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49dc0b3590 0x7f49dc14c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e2fe2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc15bb00 0x7f49dc15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e2fe2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc0cec60 con 0x7f49dc0afc00 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e2fe2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc0b42f0 con 0x7f49dc15bb00 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e2fe2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f49dc0cede0 con 0x7f49dc0b3590 2024-04-17T06:09:57.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc1529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54914/0 (socket says 172.21.15.135:54914) 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 -- 172.21.15.135:0/3594316201 learned_addr learned my addr 172.21.15.135:0/3594316201 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e157f640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc15bb00 0x7f49dc15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 -- 172.21.15.135:0/3594316201 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49dc0b3590 msgr2=0x7f49dc14c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49dc0b3590 0x7f49dc14c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 -- 172.21.15.135:0/3594316201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc15bb00 msgr2=0x7f49dc15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc15bb00 0x7f49dc15a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 -- 172.21.15.135:0/3594316201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc0ae310 con 0x7f49dc0afc00 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49e0d7e640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc1529a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c460 tx=0x7f49d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.515+0000 7f49d17fa640 1 -- 172.21.15.135:0/3594316201 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80074e0 con 0x7f49dc0afc00 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.516+0000 7f49d17fa640 1 -- 172.21.15.135:0/3594316201 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d80047e0 con 0x7f49dc0afc00 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.516+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc0ae5f0 con 0x7f49dc0afc00 2024-04-17T06:09:57.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.516+0000 7f49d17fa640 1 -- 172.21.15.135:0/3594316201 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8004bd0 con 0x7f49dc0afc00 2024-04-17T06:09:57.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.516+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc0b1f60 con 0x7f49dc0afc00 2024-04-17T06:09:57.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.516+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49dc074940 con 0x7f49dc0afc00 2024-04-17T06:09:57.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.517+0000 7f49d17fa640 1 -- 172.21.15.135:0/3594316201 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49d8011070 con 0x7f49dc0afc00 2024-04-17T06:09:57.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.517+0000 7f49d17fa640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c806c8f0 0x7f49c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.518+0000 7f49d17fa640 1 -- 172.21.15.135:0/3594316201 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f49d808baf0 con 0x7f49dc0afc00 2024-04-17T06:09:57.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.518+0000 7f49d3fff640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c806c8f0 0x7f49c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.518+0000 7f49d3fff640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c806c8f0 0x7f49c806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49dc159d40 tx=0x7f49cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.522+0000 7f49d17fa640 1 -- 172.21.15.135:0/3594316201 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49d8059be0 con 0x7f49dc0afc00 2024-04-17T06:09:57.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.698+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49dc158d20 con 0x7f49dc0afc00 2024-04-17T06:09:57.698 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.699+0000 7f49d17fa640 1 -- 172.21.15.135:0/3594316201 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f49d80594e0 con 0x7f49dc0afc00 2024-04-17T06:09:57.698 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:57.698 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:57.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c806c8f0 msgr2=0x7f49c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c806c8f0 0x7f49c806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49dc159d40 tx=0x7f49cc008040 comp rx=0 tx=0).stop 2024-04-17T06:09:57.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 msgr2=0x7f49dc1529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc1529a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c460 tx=0x7f49d800c920 comp rx=0 tx=0).stop 2024-04-17T06:09:57.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 shutdown_connections 2024-04-17T06:09:57.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49dc15bb00 0x7f49dc15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49dc0b3590 0x7f49dc14c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.700 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f49c806c8f0 0x7f49c806edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 --2- 172.21.15.135:0/3594316201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49dc0afc00 0x7f49dc1529a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 >> 172.21.15.135:0/3594316201 conn(0x7f49dc0d0520 msgr2=0x7f49dc06b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:57.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 shutdown_connections 2024-04-17T06:09:57.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.701+0000 7f49e2fe2640 1 -- 172.21.15.135:0/3594316201 wait complete. 2024-04-17T06:09:57.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:09:57.712 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:57.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.819+0000 7f6215fa8640 1 Processor -- start 2024-04-17T06:09:57.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.819+0000 7f6215fa8640 1 -- start start 2024-04-17T06:09:57.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f6215fa8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f621015bb00 0x7f62100b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f6215fa8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 0x7f62100af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f6215fa8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62100afc00 0x7f6210071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f6215fa8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f62100b0080 con 0x7f621015bb00 2024-04-17T06:09:57.820 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f6215fa8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f62100b4660 con 0x7f62100afc00 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f6215fa8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f62100b3a50 con 0x7f62100b3590 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f621015bb00 0x7f62100b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620f7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f621015bb00 0x7f62100b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54926/0 (socket says 172.21.15.135:54926) 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620f7fe640 1 -- 172.21.15.135:0/3724569402 learned_addr learned my addr 172.21.15.135:0/3724569402 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620ffff640 1 --2- 172.21.15.135:0/3724569402 >> v2:172.21.15.190:3300/0 conn(0x7f62100afc00 0x7f6210071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620effd640 1 --2- 172.21.15.135:0/3724569402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 0x7f62100af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620f7fe640 1 -- 172.21.15.135:0/3724569402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 msgr2=0x7f62100af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620f7fe640 1 --2- 172.21.15.135:0/3724569402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 0x7f62100af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620f7fe640 1 -- 172.21.15.135:0/3724569402 >> v2:172.21.15.190:3300/0 conn(0x7f62100afc00 msgr2=0x7f6210071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620f7fe640 1 --2- 172.21.15.135:0/3724569402 >> v2:172.21.15.190:3300/0 conn(0x7f62100afc00 0x7f6210071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.820+0000 7f620f7fe640 1 -- 172.21.15.135:0/3724569402 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62100b3d30 con 0x7f621015bb00 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.821+0000 7f620f7fe640 1 --2- 172.21.15.135:0/3724569402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f621015bb00 0x7f62100b5150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f62100718d0 tx=0x7f620002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc7715d5eac5f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.821 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.821+0000 7f620e7fc640 1 -- 172.21.15.135:0/3724569402 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620002ca70 con 0x7f621015bb00 2024-04-17T06:09:57.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.821+0000 7f620e7fc640 1 -- 172.21.15.135:0/3724569402 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620002cc10 con 0x7f621015bb00 2024-04-17T06:09:57.822 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.821+0000 7f620e7fc640 1 -- 172.21.15.135:0/3724569402 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62000343f0 con 0x7f621015bb00 2024-04-17T06:09:57.824 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 -- 172.21.15.135:0/3724569402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f621015bb00 msgr2=0x7f62100b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3724569402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f621015bb00 0x7f62100b5150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f62100718d0 tx=0x7f620002bed0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 -- 172.21.15.135:0/3724569402 shutdown_connections 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3724569402 >> v2:172.21.15.190:3300/0 conn(0x7f62100afc00 0x7f6210071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3724569402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 0x7f62100af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3724569402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f621015bb00 0x7f62100b5150 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 -- 172.21.15.135:0/3724569402 >> 172.21.15.135:0/3724569402 conn(0x7f62100d0520 msgr2=0x7f62100d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 -- 172.21.15.135:0/3724569402 shutdown_connections 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.822+0000 7f6215fa8640 1 -- 172.21.15.135:0/3724569402 wait complete. 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.823+0000 7f6215fa8640 1 Processor -- start 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.823+0000 7f6215fa8640 1 -- start start 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f6215fa8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100afc00 0x7f6210140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f6215fa8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 0x7f621014e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.825 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f6215fa8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015bb00 0x7f62100cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f6215fa8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62100b5690 con 0x7f62100afc00 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f6215fa8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f62100b42f0 con 0x7f621015bb00 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f6215fa8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f62100b5810 con 0x7f62100b3590 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100afc00 0x7f6210140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100afc00 0x7f6210140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54930/0 (socket says 172.21.15.135:54930) 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620f7fe640 1 -- 172.21.15.135:0/3654786103 learned_addr learned my addr 172.21.15.135:0/3654786103 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620f7fe640 1 -- 172.21.15.135:0/3654786103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 msgr2=0x7f621014e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620effd640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 0x7f621014e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620ffff640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015bb00 0x7f62100cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620f7fe640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 0x7f621014e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620f7fe640 1 -- 172.21.15.135:0/3654786103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015bb00 msgr2=0x7f62100cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:57.826 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620f7fe640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015bb00 0x7f62100cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:57.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.824+0000 7f620f7fe640 1 -- 172.21.15.135:0/3654786103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62101507d0 con 0x7f62100afc00 2024-04-17T06:09:57.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.825+0000 7f620f7fe640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100afc00 0x7f6210140fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6200009f40 tx=0x7f6200004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.825+0000 7f61f7fff640 1 -- 172.21.15.135:0/3654786103 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200043020 con 0x7f62100afc00 2024-04-17T06:09:57.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.825+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621014f110 con 0x7f62100afc00 2024-04-17T06:09:57.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.825+0000 7f61f7fff640 1 -- 172.21.15.135:0/3654786103 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f620003e450 con 0x7f62100afc00 2024-04-17T06:09:57.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.825+0000 7f61f7fff640 1 -- 172.21.15.135:0/3654786103 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200009510 con 0x7f62100afc00 2024-04-17T06:09:57.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.825+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621014f6d0 con 0x7f62100afc00 2024-04-17T06:09:57.827 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.826+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6210068a20 con 0x7f62100afc00 2024-04-17T06:09:57.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.826+0000 7f61f7fff640 1 -- 172.21.15.135:0/3654786103 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6200046020 con 0x7f62100afc00 2024-04-17T06:09:57.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.827+0000 7f61f7fff640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f61fc06c8f0 0x7f61fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:57.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.827+0000 7f620effd640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f61fc06c8f0 0x7f61fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:57.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.827+0000 7f61f7fff640 1 -- 172.21.15.135:0/3654786103 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f62000ae6e0 con 0x7f62100afc00 2024-04-17T06:09:57.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.827+0000 7f620effd640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f61fc06c8f0 0x7f61fc06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62100cf9e0 tx=0x7f6204008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:57.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:57.831+0000 7f61f7fff640 1 -- 172.21.15.135:0/3654786103 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6210068a20 con 0x7f62100afc00 2024-04-17T06:09:58.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.008+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6210074940 con 0x7f62100afc00 2024-04-17T06:09:58.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.008+0000 7f61f7fff640 1 -- 172.21.15.135:0/3654786103 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f620004d030 con 0x7f62100afc00 2024-04-17T06:09:58.008 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f61fc06c8f0 msgr2=0x7f61fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f61fc06c8f0 0x7f61fc06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62100cf9e0 tx=0x7f6204008040 comp rx=0 tx=0).stop 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100afc00 msgr2=0x7f6210140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100afc00 0x7f6210140fc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6200009f40 tx=0x7f6200004a40 comp rx=0 tx=0).stop 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 shutdown_connections 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f621015bb00 0x7f62100cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62100b3590 0x7f621014e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f61fc06c8f0 0x7f61fc06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 --2- 172.21.15.135:0/3654786103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62100afc00 0x7f6210140fc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.010+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 >> 172.21.15.135:0/3654786103 conn(0x7f62100d0520 msgr2=0x7f62101572d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:58.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.011+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 shutdown_connections 2024-04-17T06:09:58.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.011+0000 7f6215fa8640 1 -- 172.21.15.135:0/3654786103 wait complete. 2024-04-17T06:09:58.021 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:09:58.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.128+0000 7f0cbc767640 1 Processor -- start 2024-04-17T06:09:58.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.128+0000 7f0cbc767640 1 -- start start 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbc767640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0cb406d020 0x7f0cb4070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbc767640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0cb40d03d0 0x7f0cb40708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbc767640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb415cea0 con 0x7f0cb415cac0 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbc767640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb406ef60 con 0x7f0cb40d03d0 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbc767640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb4070df0 con 0x7f0cb406d020 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbad04640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0cb40d03d0 0x7f0cb40708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbad04640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0cb40d03d0 0x7f0cb40708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54938/0 (socket says 172.21.15.135:54938) 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbad04640 1 -- 172.21.15.135:0/3262217631 learned_addr learned my addr 172.21.15.135:0/3262217631 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:58.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbad04640 1 -- 172.21.15.135:0/3262217631 >> v2:172.21.15.190:3300/0 conn(0x7f0cb406d020 msgr2=0x7f0cb4070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:58.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbad04640 1 --2- 172.21.15.135:0/3262217631 >> v2:172.21.15.190:3300/0 conn(0x7f0cb406d020 0x7f0cb4070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbad04640 1 -- 172.21.15.135:0/3262217631 >> v1:172.21.15.190:6789/0 conn(0x7f0cb415cac0 legacy=0x7f0cb406e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:58.129 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.129+0000 7f0cbad04640 1 -- 172.21.15.135:0/3262217631 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb40d0940 con 0x7f0cb40d03d0 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.130+0000 7f0cbad04640 1 --2- 172.21.15.135:0/3262217631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb40d03d0 0x7f0cb40708b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4073fa0 tx=0x7f0cac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd7e4d16681b40c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.130+0000 7f0cb9501640 1 -- 172.21.15.135:0/3262217631 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac02ca80 con 0x7f0cb40d03d0 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.130+0000 7f0cb9501640 1 -- 172.21.15.135:0/3262217631 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac02cc20 con 0x7f0cb40d03d0 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.130+0000 7f0cb9501640 1 -- 172.21.15.135:0/3262217631 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0343f0 con 0x7f0cb40d03d0 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.130+0000 7f0cbc767640 1 -- 172.21.15.135:0/3262217631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb40d03d0 msgr2=0x7f0cb40708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.130+0000 7f0cbc767640 1 --2- 172.21.15.135:0/3262217631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb40d03d0 0x7f0cb40708b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4073fa0 tx=0x7f0cac02bee0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.131+0000 7f0cbc767640 1 -- 172.21.15.135:0/3262217631 shutdown_connections 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.131+0000 7f0cbc767640 1 --2- 172.21.15.135:0/3262217631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb40d03d0 0x7f0cb40708b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.130 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.131+0000 7f0cbc767640 1 --2- 172.21.15.135:0/3262217631 >> v2:172.21.15.190:3300/0 conn(0x7f0cb406d020 0x7f0cb4070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.131+0000 7f0cbc767640 1 -- 172.21.15.135:0/3262217631 >> 172.21.15.135:0/3262217631 conn(0x7f0cb40cd7e0 msgr2=0x7f0cb40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:58.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.131+0000 7f0cbc767640 1 -- 172.21.15.135:0/3262217631 shutdown_connections 2024-04-17T06:09:58.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.131+0000 7f0cbc767640 1 -- 172.21.15.135:0/3262217631 wait complete. 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.132+0000 7f0cbc767640 1 Processor -- start 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.132+0000 7f0cbc767640 1 -- start start 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.132+0000 7f0cbc767640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cb406d020 0x7f0cb40ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.132+0000 7f0cbc767640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cb40d03d0 0x7f0cb414d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.132+0000 7f0cbc767640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb415cac0 0x7f0cb40b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbc767640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb406cb00 con 0x7f0cb415cac0 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbc767640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb406c820 con 0x7f0cb406d020 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbc767640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb40add30 con 0x7f0cb40d03d0 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb415cac0 0x7f0cb40b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb415cac0 0x7f0cb40b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54944/0 (socket says 172.21.15.135:54944) 2024-04-17T06:09:58.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 -- 172.21.15.135:0/2232538545 learned_addr learned my addr 172.21.15.135:0/2232538545 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cba503640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cb406d020 0x7f0cb40ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cb9d02640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cb40d03d0 0x7f0cb414d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 -- 172.21.15.135:0/2232538545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cb40d03d0 msgr2=0x7f0cb414d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cb40d03d0 0x7f0cb414d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 -- 172.21.15.135:0/2232538545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cb406d020 msgr2=0x7f0cb40ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cb406d020 0x7f0cb40ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 -- 172.21.15.135:0/2232538545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb414e330 con 0x7f0cb415cac0 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbad04640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb415cac0 0x7f0cb40b5780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009f40 tx=0x7f0cac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0caaffd640 1 -- 172.21.15.135:0/2232538545 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac043020 con 0x7f0cb415cac0 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0caaffd640 1 -- 172.21.15.135:0/2232538545 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac03e450 con 0x7f0cb415cac0 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb414e610 con 0x7f0cb415cac0 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0caaffd640 1 -- 172.21.15.135:0/2232538545 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac009510 con 0x7f0cb415cac0 2024-04-17T06:09:58.134 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.133+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb414ebd0 con 0x7f0cb415cac0 2024-04-17T06:09:58.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.134+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb4074940 con 0x7f0cb415cac0 2024-04-17T06:09:58.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.135+0000 7f0caaffd640 1 -- 172.21.15.135:0/2232538545 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cac046020 con 0x7f0cb415cac0 2024-04-17T06:09:58.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.135+0000 7f0caaffd640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0c9806c8f0 0x7f0c9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.135+0000 7f0caaffd640 1 -- 172.21.15.135:0/2232538545 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0cac0ae7a0 con 0x7f0cb415cac0 2024-04-17T06:09:58.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.135+0000 7f0cba503640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0c9806c8f0 0x7f0c9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.136+0000 7f0cba503640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0c9806c8f0 0x7f0c9806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0006fd0 tx=0x7f0cb0009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.140+0000 7f0caaffd640 1 -- 172.21.15.135:0/2232538545 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0cac04d030 con 0x7f0cb415cac0 2024-04-17T06:09:58.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.314+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0cb406c1f0 con 0x7f0cb415cac0 2024-04-17T06:09:58.314 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.315+0000 7f0caaffd640 1 -- 172.21.15.135:0/2232538545 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f0cac07c800 con 0x7f0cb415cac0 2024-04-17T06:09:58.314 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0c9806c8f0 msgr2=0x7f0c9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0c9806c8f0 0x7f0c9806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0006fd0 tx=0x7f0cb0009060 comp rx=0 tx=0).stop 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb415cac0 msgr2=0x7f0cb40b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb415cac0 0x7f0cb40b5780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009f40 tx=0x7f0cac004a40 comp rx=0 tx=0).stop 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 shutdown_connections 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0c9806c8f0 0x7f0c9806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cb415cac0 0x7f0cb40b5780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cb40d03d0 0x7f0cb414d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 --2- 172.21.15.135:0/2232538545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cb406d020 0x7f0cb40ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 >> 172.21.15.135:0/2232538545 conn(0x7f0cb40cd7e0 msgr2=0x7f0cb40ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:58.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 shutdown_connections 2024-04-17T06:09:58.317 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.317+0000 7f0cbc767640 1 -- 172.21.15.135:0/2232538545 wait complete. 2024-04-17T06:09:58.327 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:09:58.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.433+0000 7f20811a8640 1 Processor -- start 2024-04-17T06:09:58.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.433+0000 7f20811a8640 1 -- start start 2024-04-17T06:09:58.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f20811a8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f207c15cac0 0x7f207c06e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f20811a8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f207c15cea0 con 0x7f207c0d03d0 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f20811a8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f207c0d0940 con 0x7f207c06d020 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f20811a8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f207c06c540 con 0x7f207c15cac0 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f207ad9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f207c15cac0 0x7f207c06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f207b59e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f207c0d03d0 0x7f207c0add30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51656/0 (socket says 172.21.15.135:51656) 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f207b59e640 1 -- 172.21.15.135:0/1551013779 learned_addr learned my addr 172.21.15.135:0/1551013779 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f207ad9d640 1 -- 172.21.15.135:0/1551013779 >> v1:172.21.15.190:6789/0 conn(0x7f207c06d020 legacy=0x7f207c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f207ad9d640 1 -- 172.21.15.135:0/1551013779 >> v1:172.21.15.135:6789/0 conn(0x7f207c0d03d0 legacy=0x7f207c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f207ad9d640 1 -- 172.21.15.135:0/1551013779 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207c06c820 con 0x7f207c15cac0 2024-04-17T06:09:58.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.434+0000 7f2079d9b640 1 -- 172.21.15.135:0/1551013779 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2100300752 0 0) 0x7f207c15cea0 con 0x7f207c0d03d0 2024-04-17T06:09:58.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.435+0000 7f207ad9d640 1 --2- 172.21.15.135:0/1551013779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c15cac0 0x7f207c06e850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f206c009d90 tx=0x7f206c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b335a970c33a92ae server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.435+0000 7f2079d9b640 1 -- 172.21.15.135:0/1551013779 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c039070 con 0x7f207c15cac0 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.435+0000 7f2079d9b640 1 -- 172.21.15.135:0/1551013779 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f206c02cc60 con 0x7f207c15cac0 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.435+0000 7f2079d9b640 1 -- 172.21.15.135:0/1551013779 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c034660 con 0x7f207c15cac0 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.436+0000 7f20811a8640 1 -- 172.21.15.135:0/1551013779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c15cac0 msgr2=0x7f207c06e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.436+0000 7f20811a8640 1 --2- 172.21.15.135:0/1551013779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c15cac0 0x7f207c06e850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f206c009d90 tx=0x7f206c02beb0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.436+0000 7f20811a8640 1 -- 172.21.15.135:0/1551013779 shutdown_connections 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.436+0000 7f20811a8640 1 --2- 172.21.15.135:0/1551013779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c15cac0 0x7f207c06e850 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.436+0000 7f20811a8640 1 -- 172.21.15.135:0/1551013779 >> 172.21.15.135:0/1551013779 conn(0x7f207c0cd7e0 msgr2=0x7f207c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.436+0000 7f20811a8640 1 -- 172.21.15.135:0/1551013779 shutdown_connections 2024-04-17T06:09:58.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.436+0000 7f20811a8640 1 -- 172.21.15.135:0/1551013779 wait complete. 2024-04-17T06:09:58.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.437+0000 7f20811a8640 1 Processor -- start 2024-04-17T06:09:58.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.437+0000 7f20811a8640 1 -- start start 2024-04-17T06:09:58.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f20811a8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c06d020 0x7f207c1e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f20811a8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c0d03d0 0x7f207c1e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f20811a8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c15cac0 0x7f207c1e4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f20811a8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f207c06ef10 con 0x7f207c0d03d0 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f20811a8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f207c06ed90 con 0x7f207c15cac0 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f20811a8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f207c070a80 con 0x7f207c06d020 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c0d03d0 0x7f207c1e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c0d03d0 0x7f207c1e43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54962/0 (socket says 172.21.15.135:54962) 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 -- 172.21.15.135:0/976301352 learned_addr learned my addr 172.21.15.135:0/976301352 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207ad9d640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c06d020 0x7f207c1e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207b59e640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c15cac0 0x7f207c1e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 -- 172.21.15.135:0/976301352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c06d020 msgr2=0x7f207c1e98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c06d020 0x7f207c1e98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 -- 172.21.15.135:0/976301352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c15cac0 msgr2=0x7f207c1e4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c15cac0 0x7f207c1e4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 -- 172.21.15.135:0/976301352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207c1e4e60 con 0x7f207c0d03d0 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.438+0000 7f207a59c640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c0d03d0 0x7f207c1e43e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f207000c460 tx=0x7f207000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.439+0000 7f20637fe640 1 -- 172.21.15.135:0/976301352 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070016020 con 0x7f207c0d03d0 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.439+0000 7f20637fe640 1 -- 172.21.15.135:0/976301352 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2070012870 con 0x7f207c0d03d0 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.439+0000 7f20637fe640 1 -- 172.21.15.135:0/976301352 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070012c60 con 0x7f207c0d03d0 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.439+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207c0aec90 con 0x7f207c0d03d0 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.439+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f207c144920 con 0x7f207c0d03d0 2024-04-17T06:09:58.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.440+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f207c074940 con 0x7f207c0d03d0 2024-04-17T06:09:58.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.440+0000 7f20637fe640 1 -- 172.21.15.135:0/976301352 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2070019020 con 0x7f207c0d03d0 2024-04-17T06:09:58.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.441+0000 7f20637fe640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f206806c8f0 0x7f206806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.441+0000 7f207ad9d640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f206806c8f0 0x7f206806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.441+0000 7f20637fe640 1 -- 172.21.15.135:0/976301352 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f207008bc00 con 0x7f207c0d03d0 2024-04-17T06:09:58.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.441+0000 7f207ad9d640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f206806c8f0 0x7f206806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f206c036000 tx=0x7f206c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.445+0000 7f20637fe640 1 -- 172.21.15.135:0/976301352 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2070092080 con 0x7f207c0d03d0 2024-04-17T06:09:58.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.619+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f207c0cf930 con 0x7f207c0d03d0 2024-04-17T06:09:58.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.619+0000 7f20637fe640 1 -- 172.21.15.135:0/976301352 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f2070020030 con 0x7f207c0d03d0 2024-04-17T06:09:58.619 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:09:58.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f206806c8f0 msgr2=0x7f206806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f206806c8f0 0x7f206806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f206c036000 tx=0x7f206c030000 comp rx=0 tx=0).stop 2024-04-17T06:09:58.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c0d03d0 msgr2=0x7f207c1e43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c0d03d0 0x7f207c1e43e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f207000c460 tx=0x7f207000c920 comp rx=0 tx=0).stop 2024-04-17T06:09:58.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 shutdown_connections 2024-04-17T06:09:58.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c15cac0 0x7f207c1e4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f206806c8f0 0x7f206806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.621 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f207c0d03d0 0x7f207c1e43e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 --2- 172.21.15.135:0/976301352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c06d020 0x7f207c1e98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.621+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 >> 172.21.15.135:0/976301352 conn(0x7f207c0cd7e0 msgr2=0x7f207c0af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:58.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.622+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 shutdown_connections 2024-04-17T06:09:58.622 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.622+0000 7f20811a8640 1 -- 172.21.15.135:0/976301352 wait complete. 2024-04-17T06:09:58.632 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:09:58.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.739+0000 7f3760ca0640 1 Processor -- start 2024-04-17T06:09:58.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.739+0000 7f3760ca0640 1 -- start start 2024-04-17T06:09:58.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3760ca0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f375c09bbe0 0x7f375c09b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3760ca0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f375c0cd630 0x7f375c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3760ca0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f375c06db80 con 0x7f375c15ee30 2024-04-17T06:09:58.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3760ca0640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f375c06de60 con 0x7f375c09bbe0 2024-04-17T06:09:58.741 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3760ca0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f375c0cce30 con 0x7f375c0cd630 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3759d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f375c09bbe0 0x7f375c09b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f375a59c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f375c15ee30 0x7f375c09a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51670/0 (socket says 172.21.15.135:51670) 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3759d9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f375c09bbe0 0x7f375c09b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54966/0 (socket says 172.21.15.135:54966) 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f375a59c640 1 -- 172.21.15.135:0/3032762854 learned_addr learned my addr 172.21.15.135:0/3032762854 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f375ad9d640 1 --2- 172.21.15.135:0/3032762854 >> v2:172.21.15.190:3300/0 conn(0x7f375c0cd630 0x7f375c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3759d9b640 1 -- 172.21.15.135:0/3032762854 >> v2:172.21.15.190:3300/0 conn(0x7f375c0cd630 msgr2=0x7f375c0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3759d9b640 1 --2- 172.21.15.135:0/3032762854 >> v2:172.21.15.190:3300/0 conn(0x7f375c0cd630 0x7f375c0cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.740+0000 7f3759d9b640 1 -- 172.21.15.135:0/3032762854 >> v1:172.21.15.135:6789/0 conn(0x7f375c15ee30 legacy=0x7f375c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.741+0000 7f3759d9b640 1 -- 172.21.15.135:0/3032762854 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c0cd110 con 0x7f375c09bbe0 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.741+0000 7f375959a640 1 -- 172.21.15.135:0/3032762854 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189104161 0 0) 0x7f375c06db80 con 0x7f375c15ee30 2024-04-17T06:09:58.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.741+0000 7f3759d9b640 1 --2- 172.21.15.135:0/3032762854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c09bbe0 0x7f375c09b100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f375c06cb30 tx=0x7f375002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4932d1a1a9f78edd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.741+0000 7f375959a640 1 -- 172.21.15.135:0/3032762854 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750035020 con 0x7f375c09bbe0 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.741+0000 7f375959a640 1 -- 172.21.15.135:0/3032762854 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3750031830 con 0x7f375c09bbe0 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.741+0000 7f375959a640 1 -- 172.21.15.135:0/3032762854 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750031c00 con 0x7f375c09bbe0 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.742+0000 7f3760ca0640 1 -- 172.21.15.135:0/3032762854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c09bbe0 msgr2=0x7f375c09b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.742+0000 7f3760ca0640 1 --2- 172.21.15.135:0/3032762854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c09bbe0 0x7f375c09b100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f375c06cb30 tx=0x7f375002b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.742+0000 7f3760ca0640 1 -- 172.21.15.135:0/3032762854 shutdown_connections 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.742+0000 7f3760ca0640 1 --2- 172.21.15.135:0/3032762854 >> v2:172.21.15.190:3300/0 conn(0x7f375c0cd630 0x7f375c0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.742+0000 7f3760ca0640 1 --2- 172.21.15.135:0/3032762854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c09bbe0 0x7f375c09b100 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.742+0000 7f3760ca0640 1 -- 172.21.15.135:0/3032762854 >> 172.21.15.135:0/3032762854 conn(0x7f375c0b41a0 msgr2=0x7f375c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.742+0000 7f3760ca0640 1 -- 172.21.15.135:0/3032762854 shutdown_connections 2024-04-17T06:09:58.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.742+0000 7f3760ca0640 1 -- 172.21.15.135:0/3032762854 wait complete. 2024-04-17T06:09:58.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.743+0000 7f3760ca0640 1 Processor -- start 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.743+0000 7f3760ca0640 1 -- start start 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3760ca0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f375c09bbe0 0x7f375c1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3760ca0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0cd630 0x7f375c15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3760ca0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375c15ee30 0x7f375c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3760ca0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f375c06d910 con 0x7f375c0cd630 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3760ca0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f375c06d790 con 0x7f375c09bbe0 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3760ca0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f375c09b640 con 0x7f375c15ee30 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3759d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0cd630 0x7f375c15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3759d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0cd630 0x7f375c15ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54972/0 (socket says 172.21.15.135:54972) 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f3759d9b640 1 -- 172.21.15.135:0/1845384249 learned_addr learned my addr 172.21.15.135:0/1845384249 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f375a59c640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f375c09bbe0 0x7f375c1f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.744+0000 7f375ad9d640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375c15ee30 0x7f375c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f3759d9b640 1 -- 172.21.15.135:0/1845384249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375c15ee30 msgr2=0x7f375c153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f3759d9b640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375c15ee30 0x7f375c153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f3759d9b640 1 -- 172.21.15.135:0/1845384249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f375c09bbe0 msgr2=0x7f375c1f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f3759d9b640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f375c09bbe0 0x7f375c1f1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f3759d9b640 1 -- 172.21.15.135:0/1845384249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c0caad0 con 0x7f375c0cd630 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f3759d9b640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0cd630 0x7f375c15ff90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3750009890 tx=0x7f37500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f374affd640 1 -- 172.21.15.135:0/1845384249 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750041070 con 0x7f375c0cd630 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f374affd640 1 -- 172.21.15.135:0/1845384249 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375003aa30 con 0x7f375c0cd630 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c0cadb0 con 0x7f375c0cd630 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f374affd640 1 -- 172.21.15.135:0/1845384249 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750035040 con 0x7f375c0cd630 2024-04-17T06:09:58.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.745+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c0cb370 con 0x7f375c0cd630 2024-04-17T06:09:58.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.746+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375c074940 con 0x7f375c0cd630 2024-04-17T06:09:58.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.746+0000 7f374affd640 1 -- 172.21.15.135:0/1845384249 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3750038020 con 0x7f375c0cd630 2024-04-17T06:09:58.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.747+0000 7f374affd640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f373c06c8f0 0x7f373c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:58.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.747+0000 7f375a59c640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f373c06c8f0 0x7f373c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:58.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.747+0000 7f374affd640 1 -- 172.21.15.135:0/1845384249 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f37500ad810 con 0x7f375c0cd630 2024-04-17T06:09:58.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.747+0000 7f375a59c640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f373c06c8f0 0x7f373c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f374c007fd0 tx=0x7f374c007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:58.750 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.751+0000 7f374affd640 1 -- 172.21.15.135:0/1845384249 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f375007b2b0 con 0x7f375c0cd630 2024-04-17T06:09:58.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.928+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f375c0cf180 con 0x7f375c0cd630 2024-04-17T06:09:58.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.928+0000 7f374affd640 1 -- 172.21.15.135:0/1845384249 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f37500abe20 con 0x7f375c0cd630 2024-04-17T06:09:58.928 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:58.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f373c06c8f0 msgr2=0x7f373c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f373c06c8f0 0x7f373c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f374c007fd0 tx=0x7f374c007de0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0cd630 msgr2=0x7f375c15ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0cd630 0x7f375c15ff90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3750009890 tx=0x7f37500046c0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 shutdown_connections 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f375c15ee30 0x7f375c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f373c06c8f0 0x7f373c06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f375c0cd630 0x7f375c15ff90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 --2- 172.21.15.135:0/1845384249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f375c09bbe0 0x7f375c1f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 >> 172.21.15.135:0/1845384249 conn(0x7f375c0b41a0 msgr2=0x7f375c0d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 shutdown_connections 2024-04-17T06:09:58.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:58.930+0000 7f3760ca0640 1 -- 172.21.15.135:0/1845384249 wait complete. 2024-04-17T06:09:58.942 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:09:59.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.046+0000 7f66bf7fa640 1 Processor -- start 2024-04-17T06:09:59.045 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.046+0000 7f66bf7fa640 1 -- start start 2024-04-17T06:09:59.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bf7fa640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66b815bb00 0x7f66b80b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bf7fa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80afdd0 0x7f66b80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bf7fa640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80b3a50 con 0x7f66b80b3590 2024-04-17T06:09:59.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bf7fa640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f66b80b4660 con 0x7f66b815bb00 2024-04-17T06:09:59.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bf7fa640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f66b80b3d30 con 0x7f66b80afdd0 2024-04-17T06:09:59.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bd596640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66b815bb00 0x7f66b80b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bd596640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f66b815bb00 0x7f66b80b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54982/0 (socket says 172.21.15.135:54982) 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bd596640 1 -- 172.21.15.135:0/1241926258 learned_addr learned my addr 172.21.15.135:0/1241926258 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bcd95640 1 --1- 172.21.15.135:0/1241926258 >> v1:172.21.15.135:6789/0 conn(0x7f66b80b3590 0x7f66b80af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51676/0 (socket says 172.21.15.135:51676) 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bdd97640 1 --2- 172.21.15.135:0/1241926258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80afdd0 0x7f66b80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bd596640 1 -- 172.21.15.135:0/1241926258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80afdd0 msgr2=0x7f66b80ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bd596640 1 --2- 172.21.15.135:0/1241926258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80afdd0 0x7f66b80ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bd596640 1 -- 172.21.15.135:0/1241926258 >> v1:172.21.15.135:6789/0 conn(0x7f66b80b3590 legacy=0x7f66b80af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bd596640 1 -- 172.21.15.135:0/1241926258 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b80b4010 con 0x7f66b815bb00 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66affff640 1 -- 172.21.15.135:0/1241926258 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3351119861 0 0) 0x7f66b80b3a50 con 0x7f66b80b3590 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.047+0000 7f66bdd97640 1 --2- 172.21.15.135:0/1241926258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80afdd0 0x7f66b80ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.048+0000 7f66bd596640 1 --2- 172.21.15.135:0/1241926258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b80b5150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009e50 tx=0x7f66b402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7028be3145a371 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.048+0000 7f66affff640 1 -- 172.21.15.135:0/1241926258 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b402cb20 con 0x7f66b815bb00 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.048+0000 7f66affff640 1 -- 172.21.15.135:0/1241926258 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b402ccc0 con 0x7f66b815bb00 2024-04-17T06:09:59.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.048+0000 7f66affff640 1 -- 172.21.15.135:0/1241926258 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4034610 con 0x7f66b815bb00 2024-04-17T06:09:59.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.049+0000 7f66bf7fa640 1 -- 172.21.15.135:0/1241926258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 msgr2=0x7f66b80b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.049+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/1241926258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b80b5150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009e50 tx=0x7f66b402bf80 comp rx=0 tx=0).stop 2024-04-17T06:09:59.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.049+0000 7f66bf7fa640 1 -- 172.21.15.135:0/1241926258 shutdown_connections 2024-04-17T06:09:59.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.049+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/1241926258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80afdd0 0x7f66b80ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.049+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/1241926258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b80b5150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.049+0000 7f66bf7fa640 1 -- 172.21.15.135:0/1241926258 >> 172.21.15.135:0/1241926258 conn(0x7f66b80d0520 msgr2=0x7f66b80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:59.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.049+0000 7f66bf7fa640 1 -- 172.21.15.135:0/1241926258 shutdown_connections 2024-04-17T06:09:59.050 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.049+0000 7f66bf7fa640 1 -- 172.21.15.135:0/1241926258 wait complete. 2024-04-17T06:09:59.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.050+0000 7f66bf7fa640 1 Processor -- start 2024-04-17T06:09:59.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.050+0000 7f66bf7fa640 1 -- start start 2024-04-17T06:09:59.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bf7fa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80afdd0 0x7f66b813f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bf7fa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b3590 0x7f66b81f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bf7fa640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b81420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bf7fa640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80b5810 con 0x7f66b815bb00 2024-04-17T06:09:59.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bf7fa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f66b80b5690 con 0x7f66b80afdd0 2024-04-17T06:09:59.053 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bf7fa640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f66b80cec60 con 0x7f66b80b3590 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b81420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b81420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54994/0 (socket says 172.21.15.135:54994) 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 -- 172.21.15.135:0/3308274480 learned_addr learned my addr 172.21.15.135:0/3308274480 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 -- 172.21.15.135:0/3308274480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b3590 msgr2=0x7f66b81f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bcd95640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b3590 0x7f66b81f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b3590 0x7f66b81f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 -- 172.21.15.135:0/3308274480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80afdd0 msgr2=0x7f66b813f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80afdd0 0x7f66b813f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 -- 172.21.15.135:0/3308274480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b81425e0 con 0x7f66b815bb00 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66bdd97640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b81420a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c7f0 tx=0x7f66b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66adffb640 1 -- 172.21.15.135:0/3308274480 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0015070 con 0x7f66b815bb00 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.051+0000 7f66adffb640 1 -- 172.21.15.135:0/3308274480 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b0007820 con 0x7f66b815bb00 2024-04-17T06:09:59.054 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.052+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b8142860 con 0x7f66b815bb00 2024-04-17T06:09:59.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.052+0000 7f66adffb640 1 -- 172.21.15.135:0/3308274480 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0007c50 con 0x7f66b815bb00 2024-04-17T06:09:59.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.052+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b8142e20 con 0x7f66b815bb00 2024-04-17T06:09:59.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.053+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b8074940 con 0x7f66b815bb00 2024-04-17T06:09:59.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.053+0000 7f66adffb640 1 -- 172.21.15.135:0/3308274480 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66b0018070 con 0x7f66b815bb00 2024-04-17T06:09:59.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.053+0000 7f66adffb640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c8f0 0x7f669c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.054+0000 7f66bd596640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c8f0 0x7f669c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.054+0000 7f66adffb640 1 -- 172.21.15.135:0/3308274480 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f66b008bfc0 con 0x7f66b815bb00 2024-04-17T06:09:59.055 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.054+0000 7f66bd596640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c8f0 0x7f669c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f66b4035000 tx=0x7f66b402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.057 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.058+0000 7f66adffb640 1 -- 172.21.15.135:0/3308274480 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66b81425e0 con 0x7f66b815bb00 2024-04-17T06:09:59.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.234+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66b80b1fb0 con 0x7f66b815bb00 2024-04-17T06:09:59.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.235+0000 7f66adffb640 1 -- 172.21.15.135:0/3308274480 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18562 (secure 0 0 0) 0x7f66b005a0b0 con 0x7f66b815bb00 2024-04-17T06:09:59.234 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:09:59.234 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":24,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:09:25.017267+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:09:59.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c8f0 msgr2=0x7f669c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c8f0 0x7f669c06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f66b4035000 tx=0x7f66b402e000 comp rx=0 tx=0).stop 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 msgr2=0x7f66b81420a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b81420a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c7f0 tx=0x7f66b000ccb0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 shutdown_connections 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f669c06c8f0 0x7f669c06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66b815bb00 0x7f66b81420a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66b80b3590 0x7f66b81f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 --2- 172.21.15.135:0/3308274480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f66b80afdd0 0x7f66b813f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 >> 172.21.15.135:0/3308274480 conn(0x7f66b80d0520 msgr2=0x7f66b80cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 shutdown_connections 2024-04-17T06:09:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.237+0000 7f66bf7fa640 1 -- 172.21.15.135:0/3308274480 wait complete. 2024-04-17T06:09:59.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:09:59.248 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:09:59.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.355+0000 7f1b12e60640 1 Processor -- start 2024-04-17T06:09:59.355 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.355+0000 7f1b12e60640 1 -- start start 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.356+0000 7f1b12e60640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1b0c09bbe0 0x7f1b0c09b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.356+0000 7f1b12e60640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b0c06db80 con 0x7f1b0c0cd630 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.356+0000 7f1b12e60640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b0c06de60 con 0x7f1b0c15ee30 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.356+0000 7f1b12e60640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1b0c0cce30 con 0x7f1b0c09bbe0 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.356+0000 7f1b0bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1b0c09bbe0 0x7f1b0c09b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.356+0000 7f1b113fd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f1b0c0cd630 0x7f1b0c0cda60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51686/0 (socket says 172.21.15.135:51686) 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.356+0000 7f1b0bfff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1b0c09bbe0 0x7f1b0c09b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55010/0 (socket says 172.21.15.135:55010) 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.356+0000 7f1b113fd640 1 -- 172.21.15.135:0/3721214834 learned_addr learned my addr 172.21.15.135:0/3721214834 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.357+0000 7f1b0bfff640 1 -- 172.21.15.135:0/3721214834 >> v1:172.21.15.190:6789/0 conn(0x7f1b0c15ee30 legacy=0x7f1b0c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:59.356 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.357+0000 7f1b0bfff640 1 -- 172.21.15.135:0/3721214834 >> v1:172.21.15.135:6789/0 conn(0x7f1b0c0cd630 legacy=0x7f1b0c0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.357 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.357+0000 7f1b0bfff640 1 -- 172.21.15.135:0/3721214834 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c0cd110 con 0x7f1b0c09bbe0 2024-04-17T06:09:59.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.357+0000 7f1b0bfff640 1 --2- 172.21.15.135:0/3721214834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c09bbe0 0x7f1b0c09b100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b000097a0 tx=0x7f1b0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb45fa8771848f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.357+0000 7f1b0b7fe640 1 -- 172.21.15.135:0/3721214834 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00035020 con 0x7f1b0c09bbe0 2024-04-17T06:09:59.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b0b7fe640 1 -- 172.21.15.135:0/3721214834 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b00031830 con 0x7f1b0c09bbe0 2024-04-17T06:09:59.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b0b7fe640 1 -- 172.21.15.135:0/3721214834 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00031c00 con 0x7f1b0c09bbe0 2024-04-17T06:09:59.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b12e60640 1 -- 172.21.15.135:0/3721214834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c09bbe0 msgr2=0x7f1b0c09b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b12e60640 1 --2- 172.21.15.135:0/3721214834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c09bbe0 0x7f1b0c09b100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b000097a0 tx=0x7f1b0002b8d0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.358 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b12e60640 1 -- 172.21.15.135:0/3721214834 shutdown_connections 2024-04-17T06:09:59.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b12e60640 1 --2- 172.21.15.135:0/3721214834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c09bbe0 0x7f1b0c09b100 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b12e60640 1 -- 172.21.15.135:0/3721214834 >> 172.21.15.135:0/3721214834 conn(0x7f1b0c0b41a0 msgr2=0x7f1b0c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:59.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b12e60640 1 -- 172.21.15.135:0/3721214834 shutdown_connections 2024-04-17T06:09:59.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.358+0000 7f1b12e60640 1 -- 172.21.15.135:0/3721214834 wait complete. 2024-04-17T06:09:59.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.359+0000 7f1b12e60640 1 Processor -- start 2024-04-17T06:09:59.359 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.359+0000 7f1b12e60640 1 -- start start 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b12e60640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b0c09bbe0 0x7f1b0c159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b12e60640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b0c0cd630 0x7f1b0c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b12e60640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c15ee30 0x7f1b0c0b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b12e60640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c06d910 con 0x7f1b0c15ee30 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b12e60640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c06d790 con 0x7f1b0c09bbe0 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b12e60640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0c09b640 con 0x7f1b0c0cd630 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b113fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c15ee30 0x7f1b0c0b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b113fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c15ee30 0x7f1b0c0b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55018/0 (socket says 172.21.15.135:55018) 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b113fd640 1 -- 172.21.15.135:0/3294577134 learned_addr learned my addr 172.21.15.135:0/3294577134 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:59.361 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b0bfff640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b0c0cd630 0x7f1b0c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.360+0000 7f1b10bfc640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b0c09bbe0 0x7f1b0c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b113fd640 1 -- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b0c0cd630 msgr2=0x7f1b0c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b113fd640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b0c0cd630 0x7f1b0c15fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b113fd640 1 -- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b0c09bbe0 msgr2=0x7f1b0c159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b113fd640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b0c09bbe0 0x7f1b0c159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b113fd640 1 -- 172.21.15.135:0/3294577134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c0b66c0 con 0x7f1b0c15ee30 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b0bfff640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b0c0cd630 0x7f1b0c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b113fd640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c15ee30 0x7f1b0c0b7760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400ca80 tx=0x7f1b0400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b097fa640 1 -- 172.21.15.135:0/3294577134 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04017070 con 0x7f1b0c15ee30 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b097fa640 1 -- 172.21.15.135:0/3294577134 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b0400ac30 con 0x7f1b0c15ee30 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0c0b69a0 con 0x7f1b0c15ee30 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b097fa640 1 -- 172.21.15.135:0/3294577134 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04012400 con 0x7f1b0c15ee30 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.361+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0c0b6f60 con 0x7f1b0c15ee30 2024-04-17T06:09:59.362 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.362+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0c074940 con 0x7f1b0c15ee30 2024-04-17T06:09:59.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.362+0000 7f1b097fa640 1 -- 172.21.15.135:0/3294577134 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b0401a070 con 0x7f1b0c15ee30 2024-04-17T06:09:59.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.363+0000 7f1b097fa640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1af806c8f0 0x7f1af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.363+0000 7f1b097fa640 1 -- 172.21.15.135:0/3294577134 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1b0408ba90 con 0x7f1b0c15ee30 2024-04-17T06:09:59.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.363+0000 7f1b10bfc640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1af806c8f0 0x7f1af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.363 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.363+0000 7f1b10bfc640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1af806c8f0 0x7f1af806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c15d2b0 tx=0x7f1afc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.366 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.367+0000 7f1b097fa640 1 -- 172.21.15.135:0/3294577134 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b04059530 con 0x7f1b0c15ee30 2024-04-17T06:09:59.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.541+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b0c068bc0 con 0x7f1b0c15ee30 2024-04-17T06:09:59.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.542+0000 7f1b097fa640 1 -- 172.21.15.135:0/3294577134 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f1b0405d830 con 0x7f1b0c15ee30 2024-04-17T06:09:59.541 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:09:59.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1af806c8f0 msgr2=0x7f1af806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1af806c8f0 0x7f1af806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c15d2b0 tx=0x7f1afc009040 comp rx=0 tx=0).stop 2024-04-17T06:09:59.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c15ee30 msgr2=0x7f1b0c0b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c15ee30 0x7f1b0c0b7760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400ca80 tx=0x7f1b0400cf40 comp rx=0 tx=0).stop 2024-04-17T06:09:59.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 shutdown_connections 2024-04-17T06:09:59.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1af806c8f0 0x7f1af806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1b0c15ee30 0x7f1b0c0b7760 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b0c0cd630 0x7f1b0c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 --2- 172.21.15.135:0/3294577134 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b0c09bbe0 0x7f1b0c159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 >> 172.21.15.135:0/3294577134 conn(0x7f1b0c0b41a0 msgr2=0x7f1b0c09a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:59.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 shutdown_connections 2024-04-17T06:09:59.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.544+0000 7f1b12e60640 1 -- 172.21.15.135:0/3294577134 wait complete. 2024-04-17T06:09:59.554 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:09:59.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.662+0000 7f002dab3640 1 Processor -- start 2024-04-17T06:09:59.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.662+0000 7f002dab3640 1 -- start start 2024-04-17T06:09:59.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f002dab3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00280d0900 0x7f00280d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f002dab3640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0028073a40 0x7f002815d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f002dab3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00280d1220 con 0x7f002815cae0 2024-04-17T06:09:59.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f002dab3640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f00280cea00 con 0x7f0028073a40 2024-04-17T06:09:59.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f002dab3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f00280cec60 con 0x7f00280d0900 2024-04-17T06:09:59.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f0026ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0028073a40 0x7f002815d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f0026ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f0028073a40 0x7f002815d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55028/0 (socket says 172.21.15.135:55028) 2024-04-17T06:09:59.663 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f0026ffd640 1 -- 172.21.15.135:0/4027025428 learned_addr learned my addr 172.21.15.135:0/4027025428 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f00277fe640 1 --2- 172.21.15.135:0/4027025428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00280d0900 0x7f00280d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f0026ffd640 1 -- 172.21.15.135:0/4027025428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00280d0900 msgr2=0x7f00280d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f0026ffd640 1 --2- 172.21.15.135:0/4027025428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00280d0900 0x7f00280d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f0026ffd640 1 -- 172.21.15.135:0/4027025428 >> v1:172.21.15.190:6789/0 conn(0x7f002815cae0 legacy=0x7f00280ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.663+0000 7f0026ffd640 1 -- 172.21.15.135:0/4027025428 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00280cef40 con 0x7f0028073a40 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f0026ffd640 1 --2- 172.21.15.135:0/4027025428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0028073a40 0x7f002815d800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f001c009870 tx=0x7f001c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5432bfa64ebef698 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f00267fc640 1 -- 172.21.15.135:0/4027025428 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c034070 con 0x7f0028073a40 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f00267fc640 1 -- 172.21.15.135:0/4027025428 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001c030840 con 0x7f0028073a40 2024-04-17T06:09:59.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f00267fc640 1 -- 172.21.15.135:0/4027025428 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c030c10 con 0x7f0028073a40 2024-04-17T06:09:59.667 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f002dab3640 1 -- 172.21.15.135:0/4027025428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0028073a40 msgr2=0x7f002815d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f002dab3640 1 --2- 172.21.15.135:0/4027025428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0028073a40 0x7f002815d800 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f001c009870 tx=0x7f001c02b980 comp rx=0 tx=0).stop 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f002dab3640 1 -- 172.21.15.135:0/4027025428 shutdown_connections 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f002dab3640 1 --2- 172.21.15.135:0/4027025428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0028073a40 0x7f002815d800 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.664+0000 7f002dab3640 1 --2- 172.21.15.135:0/4027025428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00280d0900 0x7f00280d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.665+0000 7f002dab3640 1 -- 172.21.15.135:0/4027025428 >> 172.21.15.135:0/4027025428 conn(0x7f00280d15e0 msgr2=0x7f00280d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.665+0000 7f002dab3640 1 -- 172.21.15.135:0/4027025428 shutdown_connections 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.665+0000 7f002dab3640 1 -- 172.21.15.135:0/4027025428 wait complete. 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.666+0000 7f002dab3640 1 Processor -- start 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.666+0000 7f002dab3640 1 -- start start 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f002dab3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0028073a40 0x7f00280b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f002dab3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00280d0900 0x7f002814b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f002dab3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f002815cae0 0x7f002814e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.668 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f002dab3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f002815dd40 con 0x7f00280d0900 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f002dab3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f002815d2e0 con 0x7f0028073a40 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f002dab3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0028160450 con 0x7f002815cae0 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00280d0900 0x7f002814b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00280d0900 0x7f002814b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55038/0 (socket says 172.21.15.135:55038) 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 -- 172.21.15.135:0/2292470894 learned_addr learned my addr 172.21.15.135:0/2292470894 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0027fff640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f002815cae0 0x7f002814e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f00277fe640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0028073a40 0x7f00280b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 -- 172.21.15.135:0/2292470894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f002815cae0 msgr2=0x7f002814e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f002815cae0 0x7f002814e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 -- 172.21.15.135:0/2292470894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0028073a40 msgr2=0x7f00280b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.669 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0028073a40 0x7f00280b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 -- 172.21.15.135:0/2292470894 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00280b3e90 con 0x7f00280d0900 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f0026ffd640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00280d0900 0x7f002814b060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f001c000c00 tx=0x7f001c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f000ffff640 1 -- 172.21.15.135:0/2292470894 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c048070 con 0x7f00280d0900 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f000ffff640 1 -- 172.21.15.135:0/2292470894 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001c030640 con 0x7f00280d0900 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f000ffff640 1 -- 172.21.15.135:0/2292470894 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c034040 con 0x7f00280d0900 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00280b1a10 con 0x7f00280d0900 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.667+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00280b22b0 con 0x7f00280d0900 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.668+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00280b79b0 con 0x7f00280d0900 2024-04-17T06:09:59.670 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.669+0000 7f000ffff640 1 -- 172.21.15.135:0/2292470894 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f001c04b070 con 0x7f00280d0900 2024-04-17T06:09:59.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.669+0000 7f000ffff640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.669+0000 7f000ffff640 1 -- 172.21.15.135:0/2292470894 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f001c0adc70 con 0x7f00280d0900 2024-04-17T06:09:59.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.669+0000 7f00277fe640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.671 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.670+0000 7f00277fe640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f001406c8f0 0x7f001406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0018009ad0 tx=0x7f0018008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.672 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.674+0000 7f000ffff640 1 -- 172.21.15.135:0/2292470894 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00280b79b0 con 0x7f00280d0900 2024-04-17T06:09:59.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.848+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0028068a20 con 0x7f00280d0900 2024-04-17T06:09:59.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.849+0000 7f000ffff640 1 -- 172.21.15.135:0/2292470894 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f001c07bdf0 con 0x7f00280d0900 2024-04-17T06:09:59.848 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:09:59.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f001406c8f0 msgr2=0x7f001406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f001406c8f0 0x7f001406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0018009ad0 tx=0x7f0018008040 comp rx=0 tx=0).stop 2024-04-17T06:09:59.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00280d0900 msgr2=0x7f002814b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00280d0900 0x7f002814b060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f001c000c00 tx=0x7f001c004730 comp rx=0 tx=0).stop 2024-04-17T06:09:59.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 shutdown_connections 2024-04-17T06:09:59.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f002815cae0 0x7f002814e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00280d0900 0x7f002814b060 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 --2- 172.21.15.135:0/2292470894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0028073a40 0x7f00280b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 >> 172.21.15.135:0/2292470894 conn(0x7f00280d15e0 msgr2=0x7f002815a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:59.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 shutdown_connections 2024-04-17T06:09:59.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.852+0000 7f002dab3640 1 -- 172.21.15.135:0/2292470894 wait complete. 2024-04-17T06:09:59.862 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:09:59.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.970+0000 7fec0d582640 1 Processor -- start 2024-04-17T06:09:59.969 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.970+0000 7fec0d582640 1 -- start start 2024-04-17T06:09:59.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec0d582640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fec0806d6c0 0x7fec080d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.970 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec0d582640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec080ccd00 con 0x7fec08162190 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec0d582640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec0806daf0 con 0x7fec0809b100 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec0d582640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fec0806ddc0 con 0x7fec0806d6c0 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec077fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fec0806d6c0 0x7fec080d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec077fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fec0806d6c0 0x7fec080d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55050/0 (socket says 172.21.15.135:55050) 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec06ffd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fec08162190 0x7fec0809a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51702/0 (socket says 172.21.15.135:51702) 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec077fe640 1 -- 172.21.15.135:0/4211366182 learned_addr learned my addr 172.21.15.135:0/4211366182 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec077fe640 1 -- 172.21.15.135:0/4211366182 >> v1:172.21.15.190:6789/0 conn(0x7fec0809b100 legacy=0x7fec080cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec077fe640 1 -- 172.21.15.135:0/4211366182 >> v1:172.21.15.135:6789/0 conn(0x7fec08162190 legacy=0x7fec0809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec05ffb640 1 -- 172.21.15.135:0/4211366182 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3861728950 0 0) 0x7fec080ccd00 con 0x7fec08162190 2024-04-17T06:09:59.971 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.971+0000 7fec077fe640 1 -- 172.21.15.135:0/4211366182 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0806e0a0 con 0x7fec0806d6c0 2024-04-17T06:09:59.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.972+0000 7fec077fe640 1 --2- 172.21.15.135:0/4211366182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0806d6c0 0x7fec080d1cf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fec00009870 tx=0x7fec0002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbb83311856af52 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.972 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.972+0000 7fec05ffb640 1 -- 172.21.15.135:0/4211366182 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00034070 con 0x7fec0806d6c0 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.972+0000 7fec05ffb640 1 -- 172.21.15.135:0/4211366182 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec00030860 con 0x7fec0806d6c0 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.972+0000 7fec05ffb640 1 -- 172.21.15.135:0/4211366182 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00030c30 con 0x7fec0806d6c0 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.973+0000 7fec0d582640 1 -- 172.21.15.135:0/4211366182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0806d6c0 msgr2=0x7fec080d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.973+0000 7fec0d582640 1 --2- 172.21.15.135:0/4211366182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0806d6c0 0x7fec080d1cf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fec00009870 tx=0x7fec0002b9a0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.973+0000 7fec0d582640 1 -- 172.21.15.135:0/4211366182 shutdown_connections 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.973+0000 7fec0d582640 1 --2- 172.21.15.135:0/4211366182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0806d6c0 0x7fec080d1cf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.973+0000 7fec0d582640 1 -- 172.21.15.135:0/4211366182 >> 172.21.15.135:0/4211366182 conn(0x7fec080b41a0 msgr2=0x7fec080b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.973+0000 7fec0d582640 1 -- 172.21.15.135:0/4211366182 shutdown_connections 2024-04-17T06:09:59.973 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.973+0000 7fec0d582640 1 -- 172.21.15.135:0/4211366182 wait complete. 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.974+0000 7fec0d582640 1 Processor -- start 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.974+0000 7fec0d582640 1 -- start start 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec0d582640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec0806d6c0 0x7fec08159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec0d582640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0809b100 0x7fec080b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec0d582640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec08162190 0x7fec0815df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec0d582640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fec08162570 con 0x7fec0809b100 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec0d582640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fec080cd2a0 con 0x7fec0806d6c0 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec0d582640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fec080cded0 con 0x7fec08162190 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0809b100 0x7fec080b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0809b100 0x7fec080b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55052/0 (socket says 172.21.15.135:55052) 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 -- 172.21.15.135:0/3215421071 learned_addr learned my addr 172.21.15.135:0/3215421071 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:09:59.976 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec077fe640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec08162190 0x7fec0815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec06ffd640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec0806d6c0 0x7fec08159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 -- 172.21.15.135:0/3215421071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec08162190 msgr2=0x7fec0815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec08162190 0x7fec0815df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 -- 172.21.15.135:0/3215421071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec0806d6c0 msgr2=0x7fec08159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec0806d6c0 0x7fec08159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 -- 172.21.15.135:0/3215421071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0815a2f0 con 0x7fec0809b100 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.975+0000 7fec067fc640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0809b100 0x7fec080b5960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7febfc00c430 tx=0x7febfc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.976+0000 7febef7fe640 1 -- 172.21.15.135:0/3215421071 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc011790 con 0x7fec0809b100 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.976+0000 7febef7fe640 1 -- 172.21.15.135:0/3215421071 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc017070 con 0x7fec0809b100 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.976+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec080b62f0 con 0x7fec0809b100 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.976+0000 7febef7fe640 1 -- 172.21.15.135:0/3215421071 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc011bb0 con 0x7fec0809b100 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.976+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec080b68b0 con 0x7fec0809b100 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.977+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec08074940 con 0x7fec0809b100 2024-04-17T06:09:59.977 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.977+0000 7febef7fe640 1 -- 172.21.15.135:0/3215421071 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febfc01c460 con 0x7fec0809b100 2024-04-17T06:09:59.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.977+0000 7febef7fe640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7febf406c8f0 0x7febf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:09:59.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.978+0000 7febef7fe640 1 -- 172.21.15.135:0/3215421071 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7febfc08be10 con 0x7fec0809b100 2024-04-17T06:09:59.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.978+0000 7fec06ffd640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7febf406c8f0 0x7febf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:09:59.978 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.978+0000 7fec06ffd640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7febf406c8f0 0x7febf406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7febf8007ed0 tx=0x7febf8007e60 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:09:59.980 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:09:59.982+0000 7febef7fe640 1 -- 172.21.15.135:0/3215421071 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febfc059e80 con 0x7fec0809b100 2024-04-17T06:10:00.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.168+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec080d2490 con 0x7fec0809b100 2024-04-17T06:10:00.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.168+0000 7febef7fe640 1 -- 172.21.15.135:0/3215421071 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7febfc059780 con 0x7fec0809b100 2024-04-17T06:10:00.168 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:10:00.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7febf406c8f0 msgr2=0x7febf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7febf406c8f0 0x7febf406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7febf8007ed0 tx=0x7febf8007e60 comp rx=0 tx=0).stop 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0809b100 msgr2=0x7fec080b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0809b100 0x7fec080b5960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7febfc00c430 tx=0x7febfc00c8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 shutdown_connections 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec08162190 0x7fec0815df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7febf406c8f0 0x7febf406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec0809b100 0x7fec080b5960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 --2- 172.21.15.135:0/3215421071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec0806d6c0 0x7fec08159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 >> 172.21.15.135:0/3215421071 conn(0x7fec080b41a0 msgr2=0x7fec0806ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.170+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 shutdown_connections 2024-04-17T06:10:00.170 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.171+0000 7fec0d582640 1 -- 172.21.15.135:0/3215421071 wait complete. 2024-04-17T06:10:00.181 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:10:00.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.289+0000 7f726ab3d640 1 Processor -- start 2024-04-17T06:10:00.288 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.289+0000 7f726ab3d640 1 -- start start 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.289+0000 7f726ab3d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f726415bb00 0x7f72640b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.289+0000 7f726ab3d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f72640afdd0 0x7f72640ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.289+0000 7f726ab3d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72640b3a50 con 0x7f72640b3590 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.289+0000 7f726ab3d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f72640b4660 con 0x7f726415bb00 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.289+0000 7f726ab3d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f72640b3d30 con 0x7f72640afdd0 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f72688d9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f726415bb00 0x7f72640b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f7263fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f72640b3590 0x7f72640af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51714/0 (socket says 172.21.15.135:51714) 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f7263fff640 1 -- 172.21.15.135:0/880002971 learned_addr learned my addr 172.21.15.135:0/880002971 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f72690da640 1 --2- 172.21.15.135:0/880002971 >> v2:172.21.15.190:3300/0 conn(0x7f72640afdd0 0x7f72640ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f72688d9640 1 -- 172.21.15.135:0/880002971 >> v2:172.21.15.190:3300/0 conn(0x7f72640afdd0 msgr2=0x7f72640ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:00.290 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f72688d9640 1 --2- 172.21.15.135:0/880002971 >> v2:172.21.15.190:3300/0 conn(0x7f72640afdd0 0x7f72640ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f72688d9640 1 -- 172.21.15.135:0/880002971 >> v1:172.21.15.135:6789/0 conn(0x7f72640b3590 legacy=0x7f72640af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f72688d9640 1 -- 172.21.15.135:0/880002971 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72640b4010 con 0x7f726415bb00 2024-04-17T06:10:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f72637fe640 1 -- 172.21.15.135:0/880002971 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902799123 0 0) 0x7f72640b3a50 con 0x7f72640b3590 2024-04-17T06:10:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.290+0000 7f72690da640 1 --2- 172.21.15.135:0/880002971 >> v2:172.21.15.190:3300/0 conn(0x7f72640afdd0 0x7f72640ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.291+0000 7f72688d9640 1 --2- 172.21.15.135:0/880002971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640b5150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f72640b01b0 tx=0x7f725802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa994efa63bed719 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.291+0000 7f72637fe640 1 -- 172.21.15.135:0/880002971 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258035020 con 0x7f726415bb00 2024-04-17T06:10:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.291+0000 7f72637fe640 1 -- 172.21.15.135:0/880002971 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7258031640 con 0x7f726415bb00 2024-04-17T06:10:00.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.291+0000 7f72637fe640 1 -- 172.21.15.135:0/880002971 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725802d040 con 0x7f726415bb00 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.292+0000 7f726ab3d640 1 -- 172.21.15.135:0/880002971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 msgr2=0x7f72640b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.292+0000 7f726ab3d640 1 --2- 172.21.15.135:0/880002971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640b5150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f72640b01b0 tx=0x7f725802b8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.292+0000 7f726ab3d640 1 -- 172.21.15.135:0/880002971 shutdown_connections 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.292+0000 7f726ab3d640 1 --2- 172.21.15.135:0/880002971 >> v2:172.21.15.190:3300/0 conn(0x7f72640afdd0 0x7f72640ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.292+0000 7f726ab3d640 1 --2- 172.21.15.135:0/880002971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640b5150 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.292+0000 7f726ab3d640 1 -- 172.21.15.135:0/880002971 >> 172.21.15.135:0/880002971 conn(0x7f72640d0520 msgr2=0x7f72640d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.292+0000 7f726ab3d640 1 -- 172.21.15.135:0/880002971 shutdown_connections 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.292+0000 7f726ab3d640 1 -- 172.21.15.135:0/880002971 wait complete. 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.293+0000 7f726ab3d640 1 Processor -- start 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.293+0000 7f726ab3d640 1 -- start start 2024-04-17T06:10:00.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f726ab3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72640afdd0 0x7f72641524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f726ab3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72640b3590 0x7f72641529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f726ab3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f726ab3d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72640b5810 con 0x7f726415bb00 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f726ab3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f72640b5690 con 0x7f72640b3590 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f726ab3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f72640cec60 con 0x7f72640afdd0 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55080/0 (socket says 172.21.15.135:55080) 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 -- 172.21.15.135:0/361261555 learned_addr learned my addr 172.21.15.135:0/361261555 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f7263fff640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72640b3590 0x7f72641529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72688d9640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72640afdd0 0x7f72641524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 -- 172.21.15.135:0/361261555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72640afdd0 msgr2=0x7f72641524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72640afdd0 0x7f72641524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 -- 172.21.15.135:0/361261555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72640b3590 msgr2=0x7f72641529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72640b3590 0x7f72641529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 -- 172.21.15.135:0/361261555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7264154540 con 0x7f726415bb00 2024-04-17T06:10:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72690da640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640cc780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f725c007e70 tx=0x7f725c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72617fa640 1 -- 172.21.15.135:0/361261555 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c01a070 con 0x7f726415bb00 2024-04-17T06:10:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72617fa640 1 -- 172.21.15.135:0/361261555 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725c015600 con 0x7f726415bb00 2024-04-17T06:10:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7264153080 con 0x7f726415bb00 2024-04-17T06:10:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.294+0000 7f72617fa640 1 -- 172.21.15.135:0/361261555 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c0159f0 con 0x7f726415bb00 2024-04-17T06:10:00.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.295+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7264154ac0 con 0x7f726415bb00 2024-04-17T06:10:00.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.295+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72640b5480 con 0x7f726415bb00 2024-04-17T06:10:00.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.296+0000 7f72617fa640 1 -- 172.21.15.135:0/361261555 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f725c01d070 con 0x7f726415bb00 2024-04-17T06:10:00.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.296+0000 7f72617fa640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f724806c8f0 0x7f724806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.296+0000 7f72617fa640 1 -- 172.21.15.135:0/361261555 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7f725c08c0b0 con 0x7f726415bb00 2024-04-17T06:10:00.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.296+0000 7f72688d9640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f724806c8f0 0x7f724806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.297+0000 7f72688d9640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f724806c8f0 0x7f724806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7258036000 tx=0x7f725803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.299 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.301+0000 7f72617fa640 1 -- 172.21.15.135:0/361261555 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72640b5480 con 0x7f726415bb00 2024-04-17T06:10:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.476+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7264152260 con 0x7f726415bb00 2024-04-17T06:10:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.476+0000 7f72617fa640 1 -- 172.21.15.135:0/361261555 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f725c010070 con 0x7f726415bb00 2024-04-17T06:10:00.475 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f724806c8f0 msgr2=0x7f724806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f724806c8f0 0x7f724806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7258036000 tx=0x7f725803c040 comp rx=0 tx=0).stop 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 msgr2=0x7f72640cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640cc780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f725c007e70 tx=0x7f725c00d4a0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 shutdown_connections 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f724806c8f0 0x7f724806edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f726415bb00 0x7f72640cc780 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72640b3590 0x7f72641529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 --2- 172.21.15.135:0/361261555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72640afdd0 0x7f72641524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 >> 172.21.15.135:0/361261555 conn(0x7f72640d0520 msgr2=0x7f72640b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 shutdown_connections 2024-04-17T06:10:00.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.478+0000 7f726ab3d640 1 -- 172.21.15.135:0/361261555 wait complete. 2024-04-17T06:10:00.489 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:00.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.597+0000 7ffad6c63640 1 Processor -- start 2024-04-17T06:10:00.596 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.597+0000 7ffad6c63640 1 -- start start 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.598+0000 7ffad6c63640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffad009b100 0x7ffad00ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.598+0000 7ffad6c63640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad00d1360 con 0x7ffad015caa0 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.598+0000 7ffad6c63640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad00cd130 con 0x7ffad00d0820 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.598+0000 7ffad6c63640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ffad00d15f0 con 0x7ffad009b100 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.598+0000 7ffacffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ffad009b100 0x7ffad00ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.598+0000 7ffad49ff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ffad015caa0 0x7ffad009a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51726/0 (socket says 172.21.15.135:51726) 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.598+0000 7ffad49ff640 1 -- 172.21.15.135:0/1087450460 learned_addr learned my addr 172.21.15.135:0/1087450460 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.598+0000 7ffacffff640 1 -- 172.21.15.135:0/1087450460 >> v1:172.21.15.190:6789/0 conn(0x7ffad00d0820 legacy=0x7ffad00d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.599+0000 7ffacffff640 1 -- 172.21.15.135:0/1087450460 >> v1:172.21.15.135:6789/0 conn(0x7ffad015caa0 legacy=0x7ffad009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.598 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.599+0000 7ffacf7fe640 1 -- 172.21.15.135:0/1087450460 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639116838 0 0) 0x7ffad00d1360 con 0x7ffad015caa0 2024-04-17T06:10:00.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.599+0000 7ffacffff640 1 -- 172.21.15.135:0/1087450460 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad00cd3d0 con 0x7ffad009b100 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.599+0000 7ffacffff640 1 --2- 172.21.15.135:0/1087450460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad009b100 0x7ffad00ce190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffab80097a0 tx=0x7ffab802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2221fee04340f30f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.599+0000 7ffacf7fe640 1 -- 172.21.15.135:0/1087450460 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8035020 con 0x7ffad009b100 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.599+0000 7ffacf7fe640 1 -- 172.21.15.135:0/1087450460 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab8031830 con 0x7ffad009b100 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.600+0000 7ffacf7fe640 1 -- 172.21.15.135:0/1087450460 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8031c00 con 0x7ffad009b100 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.600+0000 7ffad6c63640 1 -- 172.21.15.135:0/1087450460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad009b100 msgr2=0x7ffad00ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.600+0000 7ffad6c63640 1 --2- 172.21.15.135:0/1087450460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad009b100 0x7ffad00ce190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffab80097a0 tx=0x7ffab802b8d0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.600+0000 7ffad6c63640 1 -- 172.21.15.135:0/1087450460 shutdown_connections 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.600+0000 7ffad6c63640 1 --2- 172.21.15.135:0/1087450460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad009b100 0x7ffad00ce190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.600+0000 7ffad6c63640 1 -- 172.21.15.135:0/1087450460 >> 172.21.15.135:0/1087450460 conn(0x7ffad00b4180 msgr2=0x7ffad00b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:00.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.600+0000 7ffad6c63640 1 -- 172.21.15.135:0/1087450460 shutdown_connections 2024-04-17T06:10:00.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.600+0000 7ffad6c63640 1 -- 172.21.15.135:0/1087450460 wait complete. 2024-04-17T06:10:00.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.602+0000 7ffad6c63640 1 Processor -- start 2024-04-17T06:10:00.601 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.602+0000 7ffad6c63640 1 -- start start 2024-04-17T06:10:00.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.603+0000 7ffad6c63640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad009b100 0x7ffad0149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.603+0000 7ffad6c63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad00d0820 0x7ffad00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.603+0000 7ffad6c63640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015caa0 0x7ffad014a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.603+0000 7ffad6c63640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffad00cdc70 con 0x7ffad00d0820 2024-04-17T06:10:00.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.603+0000 7ffad6c63640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ffad00cd990 con 0x7ffad009b100 2024-04-17T06:10:00.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.603+0000 7ffad6c63640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ffad00c61e0 con 0x7ffad015caa0 2024-04-17T06:10:00.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad00d0820 0x7ffad00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad00d0820 0x7ffad00caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55096/0 (socket says 172.21.15.135:55096) 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 -- 172.21.15.135:0/3480990648 learned_addr learned my addr 172.21.15.135:0/3480990648 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffad49ff640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad009b100 0x7ffad0149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffad5200640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015caa0 0x7ffad014a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 -- 172.21.15.135:0/3480990648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015caa0 msgr2=0x7ffad014a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015caa0 0x7ffad014a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 -- 172.21.15.135:0/3480990648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad009b100 msgr2=0x7ffad0149b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad009b100 0x7ffad0149b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 -- 172.21.15.135:0/3480990648 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad015dfb0 con 0x7ffad00d0820 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacffff640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad00d0820 0x7ffad00caaf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffab80098c0 tx=0x7ffab80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacd7fa640 1 -- 172.21.15.135:0/3480990648 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab803ba80 con 0x7ffad00d0820 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacd7fa640 1 -- 172.21.15.135:0/3480990648 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab803bc20 con 0x7ffad00d0820 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad015e290 con 0x7ffad00d0820 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffacd7fa640 1 -- 172.21.15.135:0/3480990648 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab80453f0 con 0x7ffad00d0820 2024-04-17T06:10:00.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.604+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad00cb690 con 0x7ffad00d0820 2024-04-17T06:10:00.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.606+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad015e7b0 con 0x7ffad00d0820 2024-04-17T06:10:00.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.606+0000 7ffacd7fa640 1 -- 172.21.15.135:0/3480990648 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffab804c060 con 0x7ffad00d0820 2024-04-17T06:10:00.606 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.607+0000 7ffacd7fa640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffab006c8f0 0x7ffab006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.607+0000 7ffacd7fa640 1 -- 172.21.15.135:0/3480990648 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7ffab80adc70 con 0x7ffad00d0820 2024-04-17T06:10:00.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.607+0000 7ffad49ff640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffab006c8f0 0x7ffab006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.607+0000 7ffad49ff640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffab006c8f0 0x7ffab006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffac8007fd0 tx=0x7ffac8007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.611+0000 7ffacd7fa640 1 -- 172.21.15.135:0/3480990648 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffab8040070 con 0x7ffad00d0820 2024-04-17T06:10:00.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.807+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffad00b8510 con 0x7ffad00d0820 2024-04-17T06:10:00.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.808+0000 7ffacd7fa640 1 -- 172.21.15.135:0/3480990648 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18950 (secure 0 0 0) 0x7ffab807bdc0 con 0x7ffad00d0820 2024-04-17T06:10:00.807 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:00.807 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":25,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:00.004183+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:00.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffab006c8f0 msgr2=0x7ffab006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffab006c8f0 0x7ffab006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffac8007fd0 tx=0x7ffac8007de0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad00d0820 msgr2=0x7ffad00caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad00d0820 0x7ffad00caaf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffab80098c0 tx=0x7ffab80046c0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 shutdown_connections 2024-04-17T06:10:00.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffad015caa0 0x7ffad014a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ffab006c8f0 0x7ffab006edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffad00d0820 0x7ffad00caaf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 --2- 172.21.15.135:0/3480990648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffad009b100 0x7ffad0149b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 >> 172.21.15.135:0/3480990648 conn(0x7ffad00b4180 msgr2=0x7ffad00701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:00.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 shutdown_connections 2024-04-17T06:10:00.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.810+0000 7ffad6c63640 1 -- 172.21.15.135:0/3480990648 wait complete. 2024-04-17T06:10:00.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:10:00.821 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:10:00.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.929+0000 7fc283e4a640 1 Processor -- start 2024-04-17T06:10:00.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.930+0000 7fc283e4a640 1 -- start start 2024-04-17T06:10:00.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.930+0000 7fc283e4a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc27c144f70 0x7fc27c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.930+0000 7fc283e4a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc27c073750 0x7fc27c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc283e4a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc27c0b75c0 con 0x7fc27c15f500 2024-04-17T06:10:00.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc283e4a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc27c0cf130 con 0x7fc27c144f70 2024-04-17T06:10:00.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc283e4a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc27c160070 con 0x7fc27c073750 2024-04-17T06:10:00.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc281be6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc27c144f70 0x7fc27c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc281be6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc27c144f70 0x7fc27c0cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55108/0 (socket says 172.21.15.135:55108) 2024-04-17T06:10:00.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc281be6640 1 -- 172.21.15.135:0/3384987791 learned_addr learned my addr 172.21.15.135:0/3384987791 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:00.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc2823e7640 1 --2- 172.21.15.135:0/3384987791 >> v2:172.21.15.190:3300/0 conn(0x7fc27c073750 0x7fc27c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc281be6640 1 -- 172.21.15.135:0/3384987791 >> v2:172.21.15.190:3300/0 conn(0x7fc27c073750 msgr2=0x7fc27c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:00.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc281be6640 1 --2- 172.21.15.135:0/3384987791 >> v2:172.21.15.190:3300/0 conn(0x7fc27c073750 0x7fc27c15f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc281be6640 1 -- 172.21.15.135:0/3384987791 >> v1:172.21.15.190:6789/0 conn(0x7fc27c15f500 legacy=0x7fc27c073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:00.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.931+0000 7fc281be6640 1 -- 172.21.15.135:0/3384987791 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c160350 con 0x7fc27c144f70 2024-04-17T06:10:00.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc281be6640 1 --2- 172.21.15.135:0/3384987791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c144f70 0x7fc27c0cebf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0b55b0 tx=0x7fc26c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08a048b483bc72a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc280be4640 1 -- 172.21.15.135:0/3384987791 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c02c9f0 con 0x7fc27c144f70 2024-04-17T06:10:00.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc280be4640 1 -- 172.21.15.135:0/3384987791 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26c02cb90 con 0x7fc27c144f70 2024-04-17T06:10:00.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc280be4640 1 -- 172.21.15.135:0/3384987791 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c0343f0 con 0x7fc27c144f70 2024-04-17T06:10:00.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc283e4a640 1 -- 172.21.15.135:0/3384987791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c144f70 msgr2=0x7fc27c0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc283e4a640 1 --2- 172.21.15.135:0/3384987791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c144f70 0x7fc27c0cebf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0b55b0 tx=0x7fc26c02be50 comp rx=0 tx=0).stop 2024-04-17T06:10:00.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc283e4a640 1 -- 172.21.15.135:0/3384987791 shutdown_connections 2024-04-17T06:10:00.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc283e4a640 1 --2- 172.21.15.135:0/3384987791 >> v2:172.21.15.190:3300/0 conn(0x7fc27c073750 0x7fc27c15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc283e4a640 1 --2- 172.21.15.135:0/3384987791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c144f70 0x7fc27c0cebf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.932+0000 7fc283e4a640 1 -- 172.21.15.135:0/3384987791 >> 172.21.15.135:0/3384987791 conn(0x7fc27c0d15e0 msgr2=0x7fc27c0d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:00.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.933+0000 7fc283e4a640 1 -- 172.21.15.135:0/3384987791 shutdown_connections 2024-04-17T06:10:00.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.933+0000 7fc283e4a640 1 -- 172.21.15.135:0/3384987791 wait complete. 2024-04-17T06:10:00.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.934+0000 7fc283e4a640 1 Processor -- start 2024-04-17T06:10:00.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.934+0000 7fc283e4a640 1 -- start start 2024-04-17T06:10:00.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.934+0000 7fc283e4a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c073750 0x7fc27c140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.934+0000 7fc283e4a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c144f70 0x7fc27c15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.934+0000 7fc283e4a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f500 0x7fc27c15dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.934+0000 7fc283e4a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c145350 con 0x7fc27c073750 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.934+0000 7fc283e4a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c0ce3f0 con 0x7fc27c144f70 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.934+0000 7fc283e4a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc27c1454d0 con 0x7fc27c15f500 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c073750 0x7fc27c140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c073750 0x7fc27c140940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55120/0 (socket says 172.21.15.135:55120) 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 -- 172.21.15.135:0/2529720343 learned_addr learned my addr 172.21.15.135:0/2529720343 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc2813e5640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c144f70 0x7fc27c15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 -- 172.21.15.135:0/2529720343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f500 msgr2=0x7fc27c15dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f500 0x7fc27c15dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 -- 172.21.15.135:0/2529720343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c144f70 msgr2=0x7fc27c15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c144f70 0x7fc27c15ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 -- 172.21.15.135:0/2529720343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c15b3b0 con 0x7fc27c073750 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc281be6640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c073750 0x7fc27c140940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc26c000f80 tx=0x7fc26c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc27a7fc640 1 -- 172.21.15.135:0/2529720343 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c043020 con 0x7fc27c073750 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc27a7fc640 1 -- 172.21.15.135:0/2529720343 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26c03e450 con 0x7fc27c073750 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c156940 con 0x7fc27c073750 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc27a7fc640 1 -- 172.21.15.135:0/2529720343 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c009510 con 0x7fc27c073750 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.935+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c156f00 con 0x7fc27c073750 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.936+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27c068a20 con 0x7fc27c073750 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.937+0000 7fc27a7fc640 1 -- 172.21.15.135:0/2529720343 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc26c046020 con 0x7fc27c073750 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.937+0000 7fc27a7fc640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc26806c8f0 0x7fc26806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.937+0000 7fc27a7fc640 1 -- 172.21.15.135:0/2529720343 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6735+0+0 (secure 0 0 0) 0x7fc26c0ae840 con 0x7fc27c073750 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.937+0000 7fc2813e5640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc26806c8f0 0x7fc26806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:00.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.938+0000 7fc2813e5640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc26806c8f0 0x7fc26806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc27c15eec0 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:00.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:00.942+0000 7fc27a7fc640 1 -- 172.21.15.135:0/2529720343 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc26c04d030 con 0x7fc27c073750 2024-04-17T06:10:01.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.122+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc27c15ba50 con 0x7fc27c073750 2024-04-17T06:10:01.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.122+0000 7fc27a7fc640 1 -- 172.21.15.135:0/2529720343 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fc26c07c870 con 0x7fc27c073750 2024-04-17T06:10:01.120 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc26806c8f0 msgr2=0x7fc26806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc26806c8f0 0x7fc26806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc27c15eec0 tx=0x7fc270009040 comp rx=0 tx=0).stop 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c073750 msgr2=0x7fc27c140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c073750 0x7fc27c140940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc26c000f80 tx=0x7fc26c004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 shutdown_connections 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27c15f500 0x7fc27c15dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc27c144f70 0x7fc27c15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc26806c8f0 0x7fc26806edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 --2- 172.21.15.135:0/2529720343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27c073750 0x7fc27c140940 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 >> 172.21.15.135:0/2529720343 conn(0x7fc27c0d15e0 msgr2=0x7fc27c0708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 shutdown_connections 2024-04-17T06:10:01.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.124+0000 7fc283e4a640 1 -- 172.21.15.135:0/2529720343 wait complete. 2024-04-17T06:10:01.134 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:10:01.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.243+0000 7f7c3fe37640 1 Processor -- start 2024-04-17T06:10:01.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.244+0000 7f7c3fe37640 1 -- start start 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.244+0000 7f7c3fe37640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c380d43d0 0x7f7c380d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.244+0000 7f7c3fe37640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7c380d1810 0x7f7c380b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.244+0000 7f7c3fe37640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c380d30a0 con 0x7f7c3815ec00 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.244+0000 7f7c3fe37640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3815efe0 con 0x7f7c380d1810 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.244+0000 7f7c3fe37640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7c380d3f40 con 0x7f7c380d43d0 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.244+0000 7f7c3e3d4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7c380d1810 0x7f7c380b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.245+0000 7f7c3e3d4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7c380d1810 0x7f7c380b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55128/0 (socket says 172.21.15.135:55128) 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.245+0000 7f7c3e3d4640 1 -- 172.21.15.135:0/2741246880 learned_addr learned my addr 172.21.15.135:0/2741246880 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.245+0000 7f7c3e3d4640 1 -- 172.21.15.135:0/2741246880 >> v2:172.21.15.190:3300/0 conn(0x7f7c380d43d0 msgr2=0x7f7c380d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.245+0000 7f7c3e3d4640 1 --2- 172.21.15.135:0/2741246880 >> v2:172.21.15.190:3300/0 conn(0x7f7c380d43d0 0x7f7c380d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.245+0000 7f7c3e3d4640 1 -- 172.21.15.135:0/2741246880 >> v1:172.21.15.190:6789/0 conn(0x7f7c3815ec00 legacy=0x7f7c380d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.245+0000 7f7c3e3d4640 1 -- 172.21.15.135:0/2741246880 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c380d3330 con 0x7f7c380d1810 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.245+0000 7f7c3e3d4640 1 --2- 172.21.15.135:0/2741246880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c380d1810 0x7f7c380b7a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7c3808ce90 tx=0x7f7c3002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3db2f3be68ac525 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3cbd1640 1 -- 172.21.15.135:0/2741246880 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30034070 con 0x7f7c380d1810 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3cbd1640 1 -- 172.21.15.135:0/2741246880 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c30030860 con 0x7f7c380d1810 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3cbd1640 1 -- 172.21.15.135:0/2741246880 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30030c30 con 0x7f7c380d1810 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3fe37640 1 -- 172.21.15.135:0/2741246880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c380d1810 msgr2=0x7f7c380b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/2741246880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c380d1810 0x7f7c380b7a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7c3808ce90 tx=0x7f7c3002b9a0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3fe37640 1 -- 172.21.15.135:0/2741246880 shutdown_connections 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/2741246880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c380d1810 0x7f7c380b7a40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/2741246880 >> v2:172.21.15.190:3300/0 conn(0x7f7c380d43d0 0x7f7c380d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3fe37640 1 -- 172.21.15.135:0/2741246880 >> 172.21.15.135:0/2741246880 conn(0x7f7c3806cc90 msgr2=0x7f7c3806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3fe37640 1 -- 172.21.15.135:0/2741246880 shutdown_connections 2024-04-17T06:10:01.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.246+0000 7f7c3fe37640 1 -- 172.21.15.135:0/2741246880 wait complete. 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.247+0000 7f7c3fe37640 1 Processor -- start 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3fe37640 1 -- start start 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3fe37640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c380d1810 0x7f7c38151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3fe37640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c380d43d0 0x7f7c380cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3fe37640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3815ec00 0x7f7c381f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3fe37640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7c380d38f0 con 0x7f7c3815ec00 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3fe37640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7c380d1c60 con 0x7f7c380d43d0 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3fe37640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7c380d3bd0 con 0x7f7c380d1810 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3e3d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3815ec00 0x7f7c381f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3e3d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3815ec00 0x7f7c381f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55140/0 (socket says 172.21.15.135:55140) 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3e3d4640 1 -- 172.21.15.135:0/955410438 learned_addr learned my addr 172.21.15.135:0/955410438 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3d3d2640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c380d43d0 0x7f7c380cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.248+0000 7f7c3dbd3640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c380d1810 0x7f7c38151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c3e3d4640 1 -- 172.21.15.135:0/955410438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c380d1810 msgr2=0x7f7c38151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c3e3d4640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c380d1810 0x7f7c38151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c3e3d4640 1 -- 172.21.15.135:0/955410438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c380d43d0 msgr2=0x7f7c380cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c3e3d4640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c380d43d0 0x7f7c380cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c3e3d4640 1 -- 172.21.15.135:0/955410438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c381523c0 con 0x7f7c3815ec00 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c3e3d4640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3815ec00 0x7f7c381f4480 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7c3002b9e0 tx=0x7f7c30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c2e7fc640 1 -- 172.21.15.135:0/955410438 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30048070 con 0x7f7c3815ec00 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c2e7fc640 1 -- 172.21.15.135:0/955410438 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c30030660 con 0x7f7c3815ec00 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c380c8940 con 0x7f7c3815ec00 2024-04-17T06:10:01.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c2e7fc640 1 -- 172.21.15.135:0/955410438 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30034040 con 0x7f7c3815ec00 2024-04-17T06:10:01.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.249+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c380c8f00 con 0x7f7c3815ec00 2024-04-17T06:10:01.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.250+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c38074940 con 0x7f7c3815ec00 2024-04-17T06:10:01.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.250+0000 7f7c2e7fc640 1 -- 172.21.15.135:0/955410438 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c3004b070 con 0x7f7c3815ec00 2024-04-17T06:10:01.250 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.251+0000 7f7c2e7fc640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.251+0000 7f7c2e7fc640 1 -- 172.21.15.135:0/955410438 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7c300ae0c0 con 0x7f7c3815ec00 2024-04-17T06:10:01.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.251+0000 7f7c3dbd3640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.251+0000 7f7c3dbd3640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009f60 tx=0x7f7c340094f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.255+0000 7f7c2e7fc640 1 -- 172.21.15.135:0/955410438 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c381523c0 con 0x7f7c3815ec00 2024-04-17T06:10:01.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.434+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c380b4780 con 0x7f7c3815ec00 2024-04-17T06:10:01.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.434+0000 7f7c2e7fc640 1 -- 172.21.15.135:0/955410438 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7c3007c180 con 0x7f7c3815ec00 2024-04-17T06:10:01.433 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.436+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c1c06c8f0 msgr2=0x7f7c1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.436+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009f60 tx=0x7f7c340094f0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.436+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3815ec00 msgr2=0x7f7c381f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.436+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3815ec00 0x7f7c381f4480 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7c3002b9e0 tx=0x7f7c30004730 comp rx=0 tx=0).stop 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.436+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 shutdown_connections 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.437+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.437+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c3815ec00 0x7f7c381f4480 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.437+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c380d43d0 0x7f7c380cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.437+0000 7f7c3fe37640 1 --2- 172.21.15.135:0/955410438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c380d1810 0x7f7c38151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.437+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 >> 172.21.15.135:0/955410438 conn(0x7f7c3806cc90 msgr2=0x7f7c380b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.437+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 shutdown_connections 2024-04-17T06:10:01.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.437+0000 7f7c3fe37640 1 -- 172.21.15.135:0/955410438 wait complete. 2024-04-17T06:10:01.446 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:10:01.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.553+0000 7f6e43c88640 1 Processor -- start 2024-04-17T06:10:01.552 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.553+0000 7f6e43c88640 1 -- start start 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.554+0000 7f6e43c88640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6e3c06d6c0 0x7f6e3c0d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.554+0000 7f6e43c88640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e3c0ccd00 con 0x7f6e3c162190 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.554+0000 7f6e43c88640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e3c06daf0 con 0x7f6e3c09b100 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.554+0000 7f6e43c88640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6e3c06ddc0 con 0x7f6e3c06d6c0 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.554+0000 7f6e42225640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6e3c06d6c0 0x7f6e3c0d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.554+0000 7f6e41a24640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f6e3c162190 0x7f6e3c09a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51734/0 (socket says 172.21.15.135:51734) 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.554+0000 7f6e42225640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6e3c06d6c0 0x7f6e3c0d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55156/0 (socket says 172.21.15.135:55156) 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.554+0000 7f6e41a24640 1 -- 172.21.15.135:0/730024285 learned_addr learned my addr 172.21.15.135:0/730024285 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.555+0000 7f6e42225640 1 -- 172.21.15.135:0/730024285 >> v1:172.21.15.190:6789/0 conn(0x7f6e3c09b100 legacy=0x7f6e3c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.555+0000 7f6e42225640 1 -- 172.21.15.135:0/730024285 >> v1:172.21.15.135:6789/0 conn(0x7f6e3c162190 legacy=0x7f6e3c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.555+0000 7f6e42225640 1 -- 172.21.15.135:0/730024285 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c06e0a0 con 0x7f6e3c06d6c0 2024-04-17T06:10:01.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.555+0000 7f6e40a22640 1 -- 172.21.15.135:0/730024285 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106543540 0 0) 0x7f6e3c0ccd00 con 0x7f6e3c162190 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.555+0000 7f6e42225640 1 --2- 172.21.15.135:0/730024285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c06d6c0 0x7f6e3c0d1cf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097a0 tx=0x7f6e3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574129a91128e5f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e40a22640 1 -- 172.21.15.135:0/730024285 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34035020 con 0x7f6e3c06d6c0 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e40a22640 1 -- 172.21.15.135:0/730024285 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e34031830 con 0x7f6e3c06d6c0 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e40a22640 1 -- 172.21.15.135:0/730024285 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34031c00 con 0x7f6e3c06d6c0 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e43c88640 1 -- 172.21.15.135:0/730024285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c06d6c0 msgr2=0x7f6e3c0d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e43c88640 1 --2- 172.21.15.135:0/730024285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c06d6c0 0x7f6e3c0d1cf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097a0 tx=0x7f6e3402b8d0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e43c88640 1 -- 172.21.15.135:0/730024285 shutdown_connections 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e43c88640 1 --2- 172.21.15.135:0/730024285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c06d6c0 0x7f6e3c0d1cf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e43c88640 1 -- 172.21.15.135:0/730024285 >> 172.21.15.135:0/730024285 conn(0x7f6e3c0b41a0 msgr2=0x7f6e3c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e43c88640 1 -- 172.21.15.135:0/730024285 shutdown_connections 2024-04-17T06:10:01.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.556+0000 7f6e43c88640 1 -- 172.21.15.135:0/730024285 wait complete. 2024-04-17T06:10:01.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e43c88640 1 Processor -- start 2024-04-17T06:10:01.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e43c88640 1 -- start start 2024-04-17T06:10:01.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e43c88640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c06d6c0 0x7f6e3c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e43c88640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3c09b100 0x7f6e3c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e43c88640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c162190 0x7f6e3c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e43c88640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c162570 con 0x7f6e3c162190 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e43c88640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0cd2a0 con 0x7f6e3c06d6c0 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e43c88640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3c0cded0 con 0x7f6e3c09b100 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e42225640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c162190 0x7f6e3c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e42225640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c162190 0x7f6e3c15df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55168/0 (socket says 172.21.15.135:55168) 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.558+0000 7f6e42225640 1 -- 172.21.15.135:0/1644897782 learned_addr learned my addr 172.21.15.135:0/1644897782 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e41a24640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c06d6c0 0x7f6e3c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e41223640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3c09b100 0x7f6e3c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e42225640 1 -- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3c09b100 msgr2=0x7f6e3c0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e42225640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3c09b100 0x7f6e3c0b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e42225640 1 -- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c06d6c0 msgr2=0x7f6e3c159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e42225640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c06d6c0 0x7f6e3c159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e42225640 1 -- 172.21.15.135:0/1644897782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c15a2f0 con 0x7f6e3c162190 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e41a24640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c06d6c0 0x7f6e3c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e42225640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c162190 0x7f6e3c15df90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098c0 tx=0x7f6e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e327fc640 1 -- 172.21.15.135:0/1644897782 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34004230 con 0x7f6e3c162190 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e327fc640 1 -- 172.21.15.135:0/1644897782 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e340043d0 con 0x7f6e3c162190 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e327fc640 1 -- 172.21.15.135:0/1644897782 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3403c990 con 0x7f6e3c162190 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c0b62f0 con 0x7f6e3c162190 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.559+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c0b68b0 con 0x7f6e3c162190 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.560+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3c074940 con 0x7f6e3c162190 2024-04-17T06:10:01.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.561+0000 7f6e327fc640 1 -- 172.21.15.135:0/1644897782 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e34038020 con 0x7f6e3c162190 2024-04-17T06:10:01.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.561+0000 7f6e327fc640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e2006c8f0 0x7f6e2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.561+0000 7f6e327fc640 1 -- 172.21.15.135:0/1644897782 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6e340ae380 con 0x7f6e3c162190 2024-04-17T06:10:01.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.561+0000 7f6e41a24640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e2006c8f0 0x7f6e2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.561+0000 7f6e41a24640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e2006c8f0 0x7f6e2006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e38007fd0 tx=0x7f6e38009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.565+0000 7f6e327fc640 1 -- 172.21.15.135:0/1644897782 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e34041070 con 0x7f6e3c162190 2024-04-17T06:10:01.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.741+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6e3c0d2490 con 0x7f6e3c162190 2024-04-17T06:10:01.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.741+0000 7f6e327fc640 1 -- 172.21.15.135:0/1644897782 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f6e34031800 con 0x7f6e3c162190 2024-04-17T06:10:01.740 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e2006c8f0 msgr2=0x7f6e2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e2006c8f0 0x7f6e2006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e38007fd0 tx=0x7f6e38009040 comp rx=0 tx=0).stop 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c162190 msgr2=0x7f6e3c15df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c162190 0x7f6e3c15df90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098c0 tx=0x7f6e340046c0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 shutdown_connections 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e2006c8f0 0x7f6e2006edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c162190 0x7f6e3c15df90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e3c09b100 0x7f6e3c0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 --2- 172.21.15.135:0/1644897782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e3c06d6c0 0x7f6e3c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 >> 172.21.15.135:0/1644897782 conn(0x7f6e3c0b41a0 msgr2=0x7f6e3c06ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 shutdown_connections 2024-04-17T06:10:01.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.743+0000 7f6e43c88640 1 -- 172.21.15.135:0/1644897782 wait complete. 2024-04-17T06:10:01.752 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:10:01.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.860+0000 7fe85b2ba640 1 Processor -- start 2024-04-17T06:10:01.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.861+0000 7fe85b2ba640 1 -- start start 2024-04-17T06:10:01.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.861+0000 7fe85b2ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8540d0900 0x7fe8540d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.861+0000 7fe85b2ba640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe854073a40 0x7fe85415d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.861+0000 7fe85b2ba640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8540d1220 con 0x7fe85415cae0 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.861+0000 7fe85b2ba640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fe8540cea00 con 0x7fe854073a40 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.861+0000 7fe85b2ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe8540cec60 con 0x7fe8540d0900 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.861+0000 7fe858855640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fe854073a40 0x7fe85415d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.861+0000 7fe859857640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe85415cae0 0x7fe8540ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51744/0 (socket says 172.21.15.135:51744) 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.862+0000 7fe859857640 1 -- 172.21.15.135:0/65938336 learned_addr learned my addr 172.21.15.135:0/65938336 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.862+0000 7fe859056640 1 --2- 172.21.15.135:0/65938336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8540d0900 0x7fe8540d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.862+0000 7fe858855640 1 -- 172.21.15.135:0/65938336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8540d0900 msgr2=0x7fe8540d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.862+0000 7fe858855640 1 --2- 172.21.15.135:0/65938336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8540d0900 0x7fe8540d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.862+0000 7fe858855640 1 -- 172.21.15.135:0/65938336 >> v1:172.21.15.135:6789/0 conn(0x7fe85415cae0 legacy=0x7fe8540ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.862+0000 7fe853fff640 1 -- 172.21.15.135:0/65938336 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3931411991 0 0) 0x7fe8540d1220 con 0x7fe85415cae0 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.862+0000 7fe858855640 1 -- 172.21.15.135:0/65938336 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8540cef40 con 0x7fe854073a40 2024-04-17T06:10:01.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe858855640 1 --2- 172.21.15.135:0/65938336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe854073a40 0x7fe85415d800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe8480097a0 tx=0x7fe84802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2eea96d19e779b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe853fff640 1 -- 172.21.15.135:0/65938336 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848035020 con 0x7fe854073a40 2024-04-17T06:10:01.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe853fff640 1 -- 172.21.15.135:0/65938336 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe848031830 con 0x7fe854073a40 2024-04-17T06:10:01.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe853fff640 1 -- 172.21.15.135:0/65938336 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848031c00 con 0x7fe854073a40 2024-04-17T06:10:01.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe85b2ba640 1 -- 172.21.15.135:0/65938336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe854073a40 msgr2=0x7fe85415d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/65938336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe854073a40 0x7fe85415d800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe8480097a0 tx=0x7fe84802b8d0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe85b2ba640 1 -- 172.21.15.135:0/65938336 shutdown_connections 2024-04-17T06:10:01.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/65938336 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe854073a40 0x7fe85415d800 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/65938336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe8540d0900 0x7fe8540d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe85b2ba640 1 -- 172.21.15.135:0/65938336 >> 172.21.15.135:0/65938336 conn(0x7fe8540d15e0 msgr2=0x7fe8540d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:01.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe85b2ba640 1 -- 172.21.15.135:0/65938336 shutdown_connections 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.863+0000 7fe85b2ba640 1 -- 172.21.15.135:0/65938336 wait complete. 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.864+0000 7fe85b2ba640 1 Processor -- start 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe85b2ba640 1 -- start start 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe85b2ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe854073a40 0x7fe8540b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe85b2ba640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8540d0900 0x7fe8540ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe85b2ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe85415cae0 0x7fe8540afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe85b2ba640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe85415dd40 con 0x7fe8540d0900 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe85b2ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe85415d2e0 con 0x7fe854073a40 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe85b2ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe854160450 con 0x7fe85415cae0 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe858855640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8540d0900 0x7fe8540ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe858855640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8540d0900 0x7fe8540ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55184/0 (socket says 172.21.15.135:55184) 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe858855640 1 -- 172.21.15.135:0/2232974651 learned_addr learned my addr 172.21.15.135:0/2232974651 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:01.866 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.865+0000 7fe859857640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe85415cae0 0x7fe8540afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe859056640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe854073a40 0x7fe8540b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe858855640 1 -- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe85415cae0 msgr2=0x7fe8540afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe858855640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe85415cae0 0x7fe8540afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe858855640 1 -- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe854073a40 msgr2=0x7fe8540b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe858855640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe854073a40 0x7fe8540b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe858855640 1 -- 172.21.15.135:0/2232974651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8540b01b0 con 0x7fe8540d0900 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe859056640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe854073a40 0x7fe8540b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe859857640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe85415cae0 0x7fe8540afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe858855640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8540d0900 0x7fe8540ae6c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe8480098c0 tx=0x7fe8480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe851ffb640 1 -- 172.21.15.135:0/2232974651 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848041070 con 0x7fe8540d0900 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe851ffb640 1 -- 172.21.15.135:0/2232974651 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84803aa30 con 0x7fe8540d0900 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8540b0460 con 0x7fe8540d0900 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe851ffb640 1 -- 172.21.15.135:0/2232974651 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848035040 con 0x7fe8540d0900 2024-04-17T06:10:01.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.866+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8540adbe0 con 0x7fe8540d0900 2024-04-17T06:10:01.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.867+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe854068a20 con 0x7fe8540d0900 2024-04-17T06:10:01.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.868+0000 7fe851ffb640 1 -- 172.21.15.135:0/2232974651 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe84804c060 con 0x7fe8540d0900 2024-04-17T06:10:01.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.868+0000 7fe851ffb640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe84006c8f0 0x7fe84006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:01.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.868+0000 7fe851ffb640 1 -- 172.21.15.135:0/2232974651 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe8480ad890 con 0x7fe8540d0900 2024-04-17T06:10:01.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.868+0000 7fe859056640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe84006c8f0 0x7fe84006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:01.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.869+0000 7fe859056640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe84006c8f0 0x7fe84006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe844009b20 tx=0x7fe8440039d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:01.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:01.872+0000 7fe851ffb640 1 -- 172.21.15.135:0/2232974651 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe84807b8c0 con 0x7fe8540d0900 2024-04-17T06:10:02.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.048+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe854073260 con 0x7fe8540d0900 2024-04-17T06:10:02.046 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.048+0000 7fe851ffb640 1 -- 172.21.15.135:0/2232974651 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fe84807b1c0 con 0x7fe8540d0900 2024-04-17T06:10:02.046 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:02.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe84006c8f0 msgr2=0x7fe84006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe84006c8f0 0x7fe84006edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe844009b20 tx=0x7fe8440039d0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8540d0900 msgr2=0x7fe8540ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8540d0900 0x7fe8540ae6c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe8480098c0 tx=0x7fe8480046c0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 shutdown_connections 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe85415cae0 0x7fe8540afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe84006c8f0 0x7fe84006edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8540d0900 0x7fe8540ae6c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 --2- 172.21.15.135:0/2232974651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe854073a40 0x7fe8540b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 >> 172.21.15.135:0/2232974651 conn(0x7fe8540d15e0 msgr2=0x7fe85415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 shutdown_connections 2024-04-17T06:10:02.049 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.050+0000 7fe85b2ba640 1 -- 172.21.15.135:0/2232974651 wait complete. 2024-04-17T06:10:02.082 INFO:tasks.cephfs_test_runner:Starting test: test_per_client_labeled_perf_counters_io (tasks.cephfs.test_admin.TestLabeledPerfCounters) That the per-client labelled perf counters depict the clients performing IO. 2024-04-17T06:10:02.083 INFO:tasks.cephfs_test_runner:test_per_client_labeled_perf_counters_io (tasks.cephfs.test_admin.TestLabeledPerfCounters) 2024-04-17T06:10:02.083 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:02.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.189+0000 7f80e4b22640 1 Processor -- start 2024-04-17T06:10:02.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.189+0000 7f80e4b22640 1 -- start start 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80e4b22640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80e015f800 0x7f80e0070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80e4b22640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e006bb30 con 0x7f80e00d0f20 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80e4b22640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e006be10 con 0x7f80e0070c90 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80e4b22640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f80e006c8f0 con 0x7f80e015f800 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80ded9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f80e00d0f20 0x7f80e00cde00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51750/0 (socket says 172.21.15.135:51750) 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80ded9d640 1 -- 172.21.15.135:0/1403837626 learned_addr learned my addr 172.21.15.135:0/1403837626 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523292994 0 0) 0x7f80e006bb30 con 0x7f80e00d0f20 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80cc003520 con 0x7f80e00d0f20 2024-04-17T06:10:02.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.190+0000 7f80de59c640 1 --2- 172.21.15.135:0/1403837626 >> v2:172.21.15.190:3300/0 conn(0x7f80e015f800 0x7f80e0070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 522007909 0 0) 0x7f80cc003520 con 0x7f80e00d0f20 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 >> v2:172.21.15.190:3300/0 conn(0x7f80e015f800 msgr2=0x7f80e0070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 --2- 172.21.15.135:0/1403837626 >> v2:172.21.15.190:3300/0 conn(0x7f80e015f800 0x7f80e0070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 >> v1:172.21.15.190:6789/0 conn(0x7f80e0070c90 legacy=0x7f80e00d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e006c0f0 con 0x7f80e00d0f20 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f80d80019f0 con 0x7f80e00d0f20 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 >> v1:172.21.15.135:6789/0 conn(0x7f80e00d0f20 legacy=0x7f80e00cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80cc0039b0 0x7f80cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80cc004800 0x7f80cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc0051c0 0x7f80cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80e006c8f0 con 0x7f80cc004800 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc00c070 con 0x7f80cc0051c0 2024-04-17T06:10:02.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc00c270 con 0x7f80cc0039b0 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80d7fff640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80cc004800 0x7f80cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ded9d640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc0051c0 0x7f80cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80de59c640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80cc0039b0 0x7f80cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80d7fff640 1 -- 172.21.15.135:0/1403837626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80cc0039b0 msgr2=0x7f80cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80d7fff640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80cc0039b0 0x7f80cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80d7fff640 1 -- 172.21.15.135:0/1403837626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc0051c0 msgr2=0x7f80cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80d7fff640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc0051c0 0x7f80cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80d7fff640 1 -- 172.21.15.135:0/1403837626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc003520 con 0x7f80cc004800 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.191+0000 7f80ded9d640 1 -- 172.21.15.135:0/1403837626 reap_dead start 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.192+0000 7f80d7fff640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80cc004800 0x7f80cc004c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f80c8001750 tx=0x7f80cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56be36966922467 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.192+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8001370 con 0x7f80cc004800 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.192+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c80052d0 con 0x7f80cc004800 2024-04-17T06:10:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.192+0000 7f80ddd9b640 1 -- 172.21.15.135:0/1403837626 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8003830 con 0x7f80cc004800 2024-04-17T06:10:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.192+0000 7f80e4b22640 1 -- 172.21.15.135:0/1403837626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80cc004800 msgr2=0x7f80cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.192+0000 7f80e4b22640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80cc004800 0x7f80cc004c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f80c8001750 tx=0x7f80cc005830 comp rx=0 tx=0).stop 2024-04-17T06:10:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.193+0000 7f80e4b22640 1 -- 172.21.15.135:0/1403837626 shutdown_connections 2024-04-17T06:10:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.193+0000 7f80e4b22640 1 --2- 172.21.15.135:0/1403837626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80cc004800 0x7f80cc004c80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.193+0000 7f80e4b22640 1 -- 172.21.15.135:0/1403837626 >> 172.21.15.135:0/1403837626 conn(0x7f80e009a9f0 msgr2=0x7f80e00b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.193+0000 7f80e4b22640 1 -- 172.21.15.135:0/1403837626 shutdown_connections 2024-04-17T06:10:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.193+0000 7f80e4b22640 1 -- 172.21.15.135:0/1403837626 wait complete. 2024-04-17T06:10:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.194+0000 7f80e4b22640 1 Processor -- start 2024-04-17T06:10:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.194+0000 7f80e4b22640 1 -- start start 2024-04-17T06:10:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80e4b22640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc004800 0x7f80e015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80e4b22640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e00afba0 0x7f80e015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80e4b22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e01f4780 0x7f80e015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80e4b22640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80e015fdf0 con 0x7f80e01f4780 2024-04-17T06:10:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80e4b22640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80e006c460 con 0x7f80cc004800 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80e4b22640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80e0150700 con 0x7f80e00afba0 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e01f4780 0x7f80e015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e01f4780 0x7f80e015bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55210/0 (socket says 172.21.15.135:55210) 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 -- 172.21.15.135:0/2415416694 learned_addr learned my addr 172.21.15.135:0/2415416694 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80de59c640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc004800 0x7f80e015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80d7fff640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e00afba0 0x7f80e015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 -- 172.21.15.135:0/2415416694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e00afba0 msgr2=0x7f80e015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e00afba0 0x7f80e015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 -- 172.21.15.135:0/2415416694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc004800 msgr2=0x7f80e015b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc004800 0x7f80e015b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 -- 172.21.15.135:0/2415416694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e0150950 con 0x7f80e01f4780 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80ded9d640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e01f4780 0x7f80e015bf80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f80e015ca80 tx=0x7f80d8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80d6ffd640 1 -- 172.21.15.135:0/2415416694 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d800ab00 con 0x7f80e01f4780 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80d6ffd640 1 -- 172.21.15.135:0/2415416694 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d8018070 con 0x7f80e01f4780 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e00aede0 con 0x7f80e01f4780 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80d6ffd640 1 -- 172.21.15.135:0/2415416694 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d80133f0 con 0x7f80e01f4780 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.195+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e00af3a0 con 0x7f80e01f4780 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.196+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e00b2690 con 0x7f80e01f4780 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.197+0000 7f80d6ffd640 1 -- 172.21.15.135:0/2415416694 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80d8013590 con 0x7f80e01f4780 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.197+0000 7f80d6ffd640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80c006c8f0 0x7f80c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.197+0000 7f80d6ffd640 1 -- 172.21.15.135:0/2415416694 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f80d808c380 con 0x7f80e01f4780 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.197+0000 7f80de59c640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80c006c8f0 0x7f80c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.198+0000 7f80de59c640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80c006c8f0 0x7f80c006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009bc0 tx=0x7f80d0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.200 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.202+0000 7f80d6ffd640 1 -- 172.21.15.135:0/2415416694 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80e00b2690 con 0x7f80e01f4780 2024-04-17T06:10:02.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.377+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80e0074940 con 0x7f80e01f4780 2024-04-17T06:10:02.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.378+0000 7f80d6ffd640 1 -- 172.21.15.135:0/2415416694 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18950 (secure 0 0 0) 0x7f80d805a3b0 con 0x7f80e01f4780 2024-04-17T06:10:02.377 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:02.377 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:01.008924+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80c006c8f0 msgr2=0x7f80c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80c006c8f0 0x7f80c006edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009bc0 tx=0x7f80d0009190 comp rx=0 tx=0).stop 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e01f4780 msgr2=0x7f80e015bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e01f4780 0x7f80e015bf80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f80e015ca80 tx=0x7f80d8007a40 comp rx=0 tx=0).stop 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 shutdown_connections 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80c006c8f0 0x7f80c006edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e01f4780 0x7f80e015bf80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e00afba0 0x7f80e015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 --2- 172.21.15.135:0/2415416694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80cc004800 0x7f80e015b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 >> 172.21.15.135:0/2415416694 conn(0x7f80e009a9f0 msgr2=0x7f80e0154c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:02.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 shutdown_connections 2024-04-17T06:10:02.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.380+0000 7f80e4b22640 1 -- 172.21.15.135:0/2415416694 wait complete. 2024-04-17T06:10:02.390 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:10:02.390 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:10:02.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.498+0000 7f0d0499d640 1 Processor -- start 2024-04-17T06:10:02.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.498+0000 7f0d0499d640 1 -- start start 2024-04-17T06:10:02.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.499+0000 7f0d0499d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d0015f800 0x7f0d00070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.499+0000 7f0d0499d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d0006bb30 con 0x7f0d00070c90 2024-04-17T06:10:02.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.499+0000 7f0d0499d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d0006be10 con 0x7f0d000d0f20 2024-04-17T06:10:02.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.499+0000 7f0d0499d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0006c8f0 con 0x7f0d0015f800 2024-04-17T06:10:02.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.499+0000 7f0cfdd9b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f0d00070c90 0x7f0d000d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51764/0 (socket says 172.21.15.135:51764) 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.499+0000 7f0cfdd9b640 1 -- 172.21.15.135:0/712532489 learned_addr learned my addr 172.21.15.135:0/712532489 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfe59c640 1 --2- 172.21.15.135:0/712532489 >> v2:172.21.15.190:3300/0 conn(0x7f0d0015f800 0x7f0d00070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 588056793 0 0) 0x7f0d0006bb30 con 0x7f0d00070c90 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cec003520 con 0x7f0d00070c90 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3288738538 0 0) 0x7f0cec003520 con 0x7f0d00070c90 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 >> v2:172.21.15.190:3300/0 conn(0x7f0d0015f800 msgr2=0x7f0d00070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 --2- 172.21.15.135:0/712532489 >> v2:172.21.15.190:3300/0 conn(0x7f0d0015f800 0x7f0d00070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 >> v1:172.21.15.190:6789/0 conn(0x7f0d000d0f20 legacy=0x7f0d000cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0006c0f0 con 0x7f0d00070c90 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f0ce8001670 con 0x7f0d00070c90 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 >> v1:172.21.15.135:6789/0 conn(0x7f0d00070c90 legacy=0x7f0d000d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cec003a00 0x7f0cec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec004800 0x7f0cec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cec0051c0 0x7f0cec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0006c8f0 con 0x7f0cec0051c0 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec00c0a0 con 0x7f0cec003a00 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec00c2a0 con 0x7f0cec004800 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfed9d640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cec0051c0 0x7f0cec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfe59c640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cec003a00 0x7f0cec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfdd9b640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec004800 0x7f0cec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfed9d640 1 -- 172.21.15.135:0/712532489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec004800 msgr2=0x7f0cec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfed9d640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec004800 0x7f0cec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfed9d640 1 -- 172.21.15.135:0/712532489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cec003a00 msgr2=0x7f0cec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfed9d640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0cec003a00 0x7f0cec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.500+0000 7f0cfed9d640 1 -- 172.21.15.135:0/712532489 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec003520 con 0x7f0cec0051c0 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.501+0000 7f0cfed9d640 1 -- 172.21.15.135:0/712532489 reap_dead start 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.501+0000 7f0cfed9d640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cec0051c0 0x7f0cec00b9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8001750 tx=0x7f0cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f3f6145973b3c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.501+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8001370 con 0x7f0cec0051c0 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.501+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8005310 con 0x7f0cec0051c0 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.501+0000 7f0cfd59a640 1 -- 172.21.15.135:0/712532489 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8003830 con 0x7f0cec0051c0 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.502+0000 7f0d0499d640 1 -- 172.21.15.135:0/712532489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cec0051c0 msgr2=0x7f0cec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.502+0000 7f0d0499d640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cec0051c0 0x7f0cec00b9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8001750 tx=0x7f0cec005830 comp rx=0 tx=0).stop 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.502+0000 7f0d0499d640 1 -- 172.21.15.135:0/712532489 shutdown_connections 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.502+0000 7f0d0499d640 1 --2- 172.21.15.135:0/712532489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cec0051c0 0x7f0cec00b9c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.502+0000 7f0d0499d640 1 -- 172.21.15.135:0/712532489 >> 172.21.15.135:0/712532489 conn(0x7f0d0009a9f0 msgr2=0x7f0d000b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.502+0000 7f0d0499d640 1 -- 172.21.15.135:0/712532489 shutdown_connections 2024-04-17T06:10:02.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.502+0000 7f0d0499d640 1 -- 172.21.15.135:0/712532489 wait complete. 2024-04-17T06:10:02.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.503+0000 7f0d0499d640 1 Processor -- start 2024-04-17T06:10:02.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.503+0000 7f0d0499d640 1 -- start start 2024-04-17T06:10:02.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0d0499d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec0051c0 0x7f0d000c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0d0499d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00159730 0x7f0d00150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0d0499d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d00159ef0 0x7f0d001574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0d0499d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0015fdf0 con 0x7f0d00159730 2024-04-17T06:10:02.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0d0499d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0006c460 con 0x7f0d00159ef0 2024-04-17T06:10:02.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0d0499d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f0d00159bb0 con 0x7f0cec0051c0 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00159730 0x7f0d00150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00159730 0x7f0d00150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55220/0 (socket says 172.21.15.135:55220) 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 -- 172.21.15.135:0/744746167 learned_addr learned my addr 172.21.15.135:0/744746167 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfed9d640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d00159ef0 0x7f0d001574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfe59c640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec0051c0 0x7f0d000c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 -- 172.21.15.135:0/744746167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec0051c0 msgr2=0x7f0d000c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec0051c0 0x7f0d000c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 -- 172.21.15.135:0/744746167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d00159ef0 msgr2=0x7f0d001574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d00159ef0 0x7f0d001574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 -- 172.21.15.135:0/744746167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d000c5230 con 0x7f0d00159730 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.504+0000 7f0cfdd9b640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00159730 0x7f0d00150b10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80072c0 tx=0x7f0ce8007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.505+0000 7f0cf6ffd640 1 -- 172.21.15.135:0/744746167 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8001620 con 0x7f0d00159730 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.505+0000 7f0cf6ffd640 1 -- 172.21.15.135:0/744746167 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8018070 con 0x7f0d00159730 2024-04-17T06:10:02.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.505+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d000c5510 con 0x7f0d00159730 2024-04-17T06:10:02.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.505+0000 7f0cf6ffd640 1 -- 172.21.15.135:0/744746167 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800fb40 con 0x7f0d00159730 2024-04-17T06:10:02.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.505+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d000c5ad0 con 0x7f0d00159730 2024-04-17T06:10:02.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.506+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d00074940 con 0x7f0d00159730 2024-04-17T06:10:02.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.506+0000 7f0cf6ffd640 1 -- 172.21.15.135:0/744746167 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ce800fce0 con 0x7f0d00159730 2024-04-17T06:10:02.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.507+0000 7f0cf6ffd640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0ce006c920 0x7f0ce006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.507+0000 7f0cfe59c640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0ce006c920 0x7f0ce006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.507+0000 7f0cf6ffd640 1 -- 172.21.15.135:0/744746167 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f0ce808bf50 con 0x7f0d00159730 2024-04-17T06:10:02.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.507+0000 7f0cfe59c640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0ce006c920 0x7f0ce006ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009fd0 tx=0x7f0cf000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.511+0000 7f0cf6ffd640 1 -- 172.21.15.135:0/744746167 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0ce8059f80 con 0x7f0d00159730 2024-04-17T06:10:02.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.686+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d00073be0 con 0x7f0d00159730 2024-04-17T06:10:02.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.686+0000 7f0cf6ffd640 1 -- 172.21.15.135:0/744746167 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f0ce8059880 con 0x7f0d00159730 2024-04-17T06:10:02.684 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0ce006c920 msgr2=0x7f0ce006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0ce006c920 0x7f0ce006ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009fd0 tx=0x7f0cf000b040 comp rx=0 tx=0).stop 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00159730 msgr2=0x7f0d00150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00159730 0x7f0d00150b10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80072c0 tx=0x7f0ce8007780 comp rx=0 tx=0).stop 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 shutdown_connections 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d00159ef0 0x7f0d001574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f0ce006c920 0x7f0ce006ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d00159730 0x7f0d00150b10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 --2- 172.21.15.135:0/744746167 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0cec0051c0 0x7f0d000c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 >> 172.21.15.135:0/744746167 conn(0x7f0d0009a9f0 msgr2=0x7f0d0006f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 shutdown_connections 2024-04-17T06:10:02.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.688+0000 7f0d0499d640 1 -- 172.21.15.135:0/744746167 wait complete. 2024-04-17T06:10:02.698 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:10:02.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.806+0000 7fe277bf0640 1 Processor -- start 2024-04-17T06:10:02.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.806+0000 7fe277bf0640 1 -- start start 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.807+0000 7fe277bf0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2700d0f20 0x7fe2701698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.807+0000 7fe277bf0640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe270070e60 con 0x7fe27015fbe0 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.807+0000 7fe277bf0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe27006bb30 con 0x7fe27015f800 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.807+0000 7fe277bf0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe27006c8f0 con 0x7fe2700d0f20 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.807+0000 7fe27518b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe27015fbe0 0x7fe2700d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51774/0 (socket says 172.21.15.135:51774) 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.807+0000 7fe27518b640 1 -- 172.21.15.135:0/4192747803 learned_addr learned my addr 172.21.15.135:0/4192747803 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.808+0000 7fe27618d640 1 --2- 172.21.15.135:0/4192747803 >> v2:172.21.15.190:3300/0 conn(0x7fe2700d0f20 0x7fe2701698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.808+0000 7fe27498a640 1 -- 172.21.15.135:0/4192747803 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464966187 0 0) 0x7fe270070e60 con 0x7fe27015fbe0 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.808+0000 7fe27498a640 1 -- 172.21.15.135:0/4192747803 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe258003520 con 0x7fe27015fbe0 2024-04-17T06:10:02.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.808+0000 7fe27618d640 1 -- 172.21.15.135:0/4192747803 >> v1:172.21.15.190:6789/0 conn(0x7fe27015f800 legacy=0x7fe270070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.808+0000 7fe27618d640 1 -- 172.21.15.135:0/4192747803 >> v1:172.21.15.135:6789/0 conn(0x7fe27015fbe0 legacy=0x7fe2700d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.808+0000 7fe27618d640 1 -- 172.21.15.135:0/4192747803 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27006be10 con 0x7fe2700d0f20 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe27618d640 1 --2- 172.21.15.135:0/4192747803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2700d0f20 0x7fe2701698c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2680097a0 tx=0x7fe26802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed4a75d9b1e90fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe27498a640 1 -- 172.21.15.135:0/4192747803 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26802f810 con 0x7fe2700d0f20 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe27498a640 1 -- 172.21.15.135:0/4192747803 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26802f9b0 con 0x7fe2700d0f20 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe27498a640 1 -- 172.21.15.135:0/4192747803 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26802fd60 con 0x7fe2700d0f20 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe277bf0640 1 -- 172.21.15.135:0/4192747803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2700d0f20 msgr2=0x7fe2701698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe277bf0640 1 --2- 172.21.15.135:0/4192747803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2700d0f20 0x7fe2701698c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2680097a0 tx=0x7fe26802b9f0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe277bf0640 1 -- 172.21.15.135:0/4192747803 shutdown_connections 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe277bf0640 1 --2- 172.21.15.135:0/4192747803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2700d0f20 0x7fe2701698c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.809+0000 7fe277bf0640 1 -- 172.21.15.135:0/4192747803 >> 172.21.15.135:0/4192747803 conn(0x7fe27009a9f0 msgr2=0x7fe2700b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.810+0000 7fe277bf0640 1 -- 172.21.15.135:0/4192747803 shutdown_connections 2024-04-17T06:10:02.808 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.810+0000 7fe277bf0640 1 -- 172.21.15.135:0/4192747803 wait complete. 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.811+0000 7fe277bf0640 1 Processor -- start 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.811+0000 7fe277bf0640 1 -- start start 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.811+0000 7fe277bf0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2700d0f20 0x7fe2701515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.811+0000 7fe277bf0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27015f800 0x7fe2700c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.811+0000 7fe277bf0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe27015fbe0 0x7fe2700c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.811+0000 7fe277bf0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe27006c460 con 0x7fe27015fbe0 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.811+0000 7fe277bf0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe2700cee10 con 0x7fe27015f800 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.811+0000 7fe277bf0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe270169e00 con 0x7fe2700d0f20 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe27015fbe0 0x7fe2700c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe27015fbe0 0x7fe2700c55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55236/0 (socket says 172.21.15.135:55236) 2024-04-17T06:10:02.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 -- 172.21.15.135:0/2698687212 learned_addr learned my addr 172.21.15.135:0/2698687212 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27518b640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27015f800 0x7fe2700c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27598c640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2700d0f20 0x7fe2701515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 -- 172.21.15.135:0/2698687212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2700d0f20 msgr2=0x7fe2701515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2700d0f20 0x7fe2701515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 -- 172.21.15.135:0/2698687212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27015f800 msgr2=0x7fe2700c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27015f800 0x7fe2700c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 -- 172.21.15.135:0/2698687212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe270154500 con 0x7fe27015fbe0 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe27618d640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe27015fbe0 0x7fe2700c55b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe2680097a0 tx=0x7fe2680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe26e7fc640 1 -- 172.21.15.135:0/2698687212 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268047070 con 0x7fe27015fbe0 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe26e7fc640 1 -- 172.21.15.135:0/2698687212 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe268004230 con 0x7fe27015fbe0 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe26e7fc640 1 -- 172.21.15.135:0/2698687212 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268037610 con 0x7fe27015fbe0 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2700c5bc0 con 0x7fe27015fbe0 2024-04-17T06:10:02.812 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.812+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2700c6180 con 0x7fe27015fbe0 2024-04-17T06:10:02.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.813+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe270159340 con 0x7fe27015fbe0 2024-04-17T06:10:02.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.814+0000 7fe26e7fc640 1 -- 172.21.15.135:0/2698687212 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe26804a070 con 0x7fe27015fbe0 2024-04-17T06:10:02.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.814+0000 7fe26e7fc640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe25c06c8f0 0x7fe25c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:02.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.814+0000 7fe26e7fc640 1 -- 172.21.15.135:0/2698687212 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fe2680ad8c0 con 0x7fe27015fbe0 2024-04-17T06:10:02.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.814+0000 7fe27598c640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe25c06c8f0 0x7fe25c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:02.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.815+0000 7fe27598c640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe25c06c8f0 0x7fe25c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe260001f10 tx=0x7fe260009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:02.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.819+0000 7fe26e7fc640 1 -- 172.21.15.135:0/2698687212 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe270159340 con 0x7fe27015fbe0 2024-04-17T06:10:02.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.992+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe270074940 con 0x7fe27015fbe0 2024-04-17T06:10:02.991 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.993+0000 7fe26e7fc640 1 -- 172.21.15.135:0/2698687212 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fe268033030 con 0x7fe27015fbe0 2024-04-17T06:10:02.991 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:02.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe25c06c8f0 msgr2=0x7fe25c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe25c06c8f0 0x7fe25c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe260001f10 tx=0x7fe260009040 comp rx=0 tx=0).stop 2024-04-17T06:10:02.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe27015fbe0 msgr2=0x7fe2700c55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:02.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe27015fbe0 0x7fe2700c55b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe2680097a0 tx=0x7fe2680046c0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 shutdown_connections 2024-04-17T06:10:02.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe25c06c8f0 0x7fe25c06edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe27015fbe0 0x7fe2700c55b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe27015f800 0x7fe2700c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 --2- 172.21.15.135:0/2698687212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2700d0f20 0x7fe2701515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:02.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 >> 172.21.15.135:0/2698687212 conn(0x7fe27009a9f0 msgr2=0x7fe2700ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:02.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 shutdown_connections 2024-04-17T06:10:02.994 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:02.995+0000 7fe277bf0640 1 -- 172.21.15.135:0/2698687212 wait complete. 2024-04-17T06:10:03.004 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:10:03.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.113+0000 7fc33d835640 1 Processor -- start 2024-04-17T06:10:03.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.113+0000 7fc33d835640 1 -- start start 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc33d835640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc3380d43d0 0x7fc3380d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc33d835640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3380d30a0 con 0x7fc3380d1810 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc33d835640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3380d3330 con 0x7fc33815ec00 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc33d835640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc33815efe0 con 0x7fc3380d43d0 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc32e7fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc3380d43d0 0x7fc3380d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc3377fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc3380d1810 0x7fc3380b7a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51776/0 (socket says 172.21.15.135:51776) 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc3377fe640 1 -- 172.21.15.135:0/2417333430 learned_addr learned my addr 172.21.15.135:0/2417333430 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc32e7fc640 1 -- 172.21.15.135:0/2417333430 >> v1:172.21.15.190:6789/0 conn(0x7fc33815ec00 legacy=0x7fc3380d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:03.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc32e7fc640 1 -- 172.21.15.135:0/2417333430 >> v1:172.21.15.135:6789/0 conn(0x7fc3380d1810 legacy=0x7fc3380b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.114+0000 7fc32e7fc640 1 -- 172.21.15.135:0/2417333430 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3380d3610 con 0x7fc3380d43d0 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.115+0000 7fc32e7fc640 1 --2- 172.21.15.135:0/2417333430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d43d0 0x7fc3380d12d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc3200097a0 tx=0x7fc32002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ad3117862f8c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.115+0000 7fc3367fc640 1 -- 172.21.15.135:0/2417333430 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320034070 con 0x7fc3380d43d0 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.115+0000 7fc3367fc640 1 -- 172.21.15.135:0/2417333430 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc320030610 con 0x7fc3380d43d0 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.115+0000 7fc3367fc640 1 -- 172.21.15.135:0/2417333430 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3200309e0 con 0x7fc3380d43d0 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.115+0000 7fc33d835640 1 -- 172.21.15.135:0/2417333430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d43d0 msgr2=0x7fc3380d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.115+0000 7fc33d835640 1 --2- 172.21.15.135:0/2417333430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d43d0 0x7fc3380d12d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc3200097a0 tx=0x7fc32002b960 comp rx=0 tx=0).stop 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.116+0000 7fc33d835640 1 -- 172.21.15.135:0/2417333430 shutdown_connections 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.116+0000 7fc33d835640 1 --2- 172.21.15.135:0/2417333430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d43d0 0x7fc3380d12d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.116+0000 7fc33d835640 1 -- 172.21.15.135:0/2417333430 >> 172.21.15.135:0/2417333430 conn(0x7fc33806cc90 msgr2=0x7fc33806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.116+0000 7fc33d835640 1 -- 172.21.15.135:0/2417333430 shutdown_connections 2024-04-17T06:10:03.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.116+0000 7fc33d835640 1 -- 172.21.15.135:0/2417333430 wait complete. 2024-04-17T06:10:03.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.117+0000 7fc33d835640 1 Processor -- start 2024-04-17T06:10:03.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.117+0000 7fc33d835640 1 -- start start 2024-04-17T06:10:03.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc33d835640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d1810 0x7fc338161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc33d835640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380d43d0 0x7fc3380cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc33d835640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc33815ec00 0x7fc338160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc33d835640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc3380d1c60 con 0x7fc3380d1810 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc33d835640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc3380d3f40 con 0x7fc3380d43d0 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc33d835640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc3380d3bd0 con 0x7fc33815ec00 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d1810 0x7fc338161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d1810 0x7fc338161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55262/0 (socket says 172.21.15.135:55262) 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 -- 172.21.15.135:0/1539239305 learned_addr learned my addr 172.21.15.135:0/1539239305 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc32e7fc640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380d43d0 0x7fc3380cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc3377fe640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc33815ec00 0x7fc338160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 -- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc33815ec00 msgr2=0x7fc338160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc33815ec00 0x7fc338160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 -- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380d43d0 msgr2=0x7fc3380cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380d43d0 0x7fc3380cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 -- 172.21.15.135:0/1539239305 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33815fd90 con 0x7fc3380d1810 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc3377fe640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc33815ec00 0x7fc338160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc336ffd640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d1810 0x7fc338161570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b3f0 tx=0x7fc32800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc334ff9640 1 -- 172.21.15.135:0/1539239305 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328015020 con 0x7fc3380d1810 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc334ff9640 1 -- 172.21.15.135:0/1539239305 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc328011830 con 0x7fc3380d1810 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc338161b50 con 0x7fc3380d1810 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc334ff9640 1 -- 172.21.15.135:0/1539239305 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328011c20 con 0x7fc3380d1810 2024-04-17T06:10:03.120 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.118+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc338160310 con 0x7fc3380d1810 2024-04-17T06:10:03.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.119+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc338074940 con 0x7fc3380d1810 2024-04-17T06:10:03.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.120+0000 7fc334ff9640 1 -- 172.21.15.135:0/1539239305 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc328018020 con 0x7fc3380d1810 2024-04-17T06:10:03.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.120+0000 7fc334ff9640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc31806c8f0 0x7fc31806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.120+0000 7fc334ff9640 1 -- 172.21.15.135:0/1539239305 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc32808be90 con 0x7fc3380d1810 2024-04-17T06:10:03.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.120+0000 7fc32e7fc640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc31806c8f0 0x7fc31806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.121+0000 7fc32e7fc640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc31806c8f0 0x7fc31806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc320036000 tx=0x7fc32003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.125+0000 7fc334ff9640 1 -- 172.21.15.135:0/1539239305 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc32801f020 con 0x7fc3380d1810 2024-04-17T06:10:03.297 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.299+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc338152310 con 0x7fc3380d1810 2024-04-17T06:10:03.298 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.300+0000 7fc334ff9640 1 -- 172.21.15.135:0/1539239305 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc3280172c0 con 0x7fc3380d1810 2024-04-17T06:10:03.298 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:10:03.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.301+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc31806c8f0 msgr2=0x7fc31806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.301+0000 7fc33d835640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc31806c8f0 0x7fc31806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc320036000 tx=0x7fc32003b040 comp rx=0 tx=0).stop 2024-04-17T06:10:03.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d1810 msgr2=0x7fc338161570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d1810 0x7fc338161570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b3f0 tx=0x7fc32800b8b0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 shutdown_connections 2024-04-17T06:10:03.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc33815ec00 0x7fc338160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3380d43d0 0x7fc3380cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc31806c8f0 0x7fc31806edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 --2- 172.21.15.135:0/1539239305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3380d1810 0x7fc338161570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 >> 172.21.15.135:0/1539239305 conn(0x7fc33806cc90 msgr2=0x7fc3380d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:03.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 shutdown_connections 2024-04-17T06:10:03.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.302+0000 7fc33d835640 1 -- 172.21.15.135:0/1539239305 wait complete. 2024-04-17T06:10:03.311 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:10:03.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.420+0000 7fd077ed3640 1 Processor -- start 2024-04-17T06:10:03.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.420+0000 7fd077ed3640 1 -- start start 2024-04-17T06:10:03.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd077ed3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd07015cc60 0x7fd0700d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd077ed3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07009b100 0x7fd0700b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd077ed3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd07015d040 con 0x7fd0700d0490 2024-04-17T06:10:03.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd077ed3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd0700d13c0 con 0x7fd07015cc60 2024-04-17T06:10:03.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd077ed3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd0700b1d20 con 0x7fd07009b100 2024-04-17T06:10:03.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd07546e640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd0700d0490 0x7fd07009a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:51790/0 (socket says 172.21.15.135:51790) 2024-04-17T06:10:03.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd07546e640 1 -- 172.21.15.135:0/1054627095 learned_addr learned my addr 172.21.15.135:0/1054627095 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:03.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd075c6f640 1 --2- 172.21.15.135:0/1054627095 >> v2:172.21.15.190:3300/0 conn(0x7fd07015cc60 0x7fd0700d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176805584 0 0) 0x7fd07015d040 con 0x7fd0700d0490 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd05c003520 con 0x7fd0700d0490 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.421+0000 7fd076470640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07009b100 0x7fd0700b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2269030123 0 0) 0x7fd05c003520 con 0x7fd0700d0490 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07009b100 msgr2=0x7fd0700b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07009b100 0x7fd0700b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 >> v2:172.21.15.190:3300/0 conn(0x7fd07015cc60 msgr2=0x7fd0700d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 --2- 172.21.15.135:0/1054627095 >> v2:172.21.15.190:3300/0 conn(0x7fd07015cc60 0x7fd0700d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0700b2000 con 0x7fd0700d0490 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fd060001670 con 0x7fd0700d0490 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 >> v1:172.21.15.135:6789/0 conn(0x7fd0700d0490 legacy=0x7fd07009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07009b100 0x7fd05c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c003f30 0x7fd05c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd05c004d40 0x7fd05c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700d13c0 con 0x7fd05c004d40 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700b1d20 con 0x7fd05c003f30 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c00b9c0 con 0x7fd07009b100 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd076470640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd05c004d40 0x7fd05c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd075c6f640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07009b100 0x7fd05c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd07546e640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c003f30 0x7fd05c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd076470640 1 -- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07009b100 msgr2=0x7fd05c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd076470640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07009b100 0x7fd05c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd076470640 1 -- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c003f30 msgr2=0x7fd05c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd076470640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd05c003f30 0x7fd05c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.422+0000 7fd076470640 1 -- 172.21.15.135:0/1054627095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c003520 con 0x7fd05c004d40 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.423+0000 7fd076470640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd05c004d40 0x7fd05c0051c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd070073310 tx=0x7fd05c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11ca1a839026503 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.423+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068039070 con 0x7fd05c004d40 2024-04-17T06:10:03.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.423+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd06802cb90 con 0x7fd05c004d40 2024-04-17T06:10:03.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.423+0000 7fd074c6d640 1 -- 172.21.15.135:0/1054627095 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068009c10 con 0x7fd05c004d40 2024-04-17T06:10:03.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.424+0000 7fd077ed3640 1 -- 172.21.15.135:0/1054627095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd05c004d40 msgr2=0x7fd05c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.424+0000 7fd077ed3640 1 --2- 172.21.15.135:0/1054627095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd05c004d40 0x7fd05c0051c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd070073310 tx=0x7fd05c005830 comp rx=0 tx=0).stop 2024-04-17T06:10:03.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.424+0000 7fd076470640 1 -- 172.21.15.135:0/1054627095 reap_dead start 2024-04-17T06:10:03.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.424+0000 7fd077ed3640 1 -- 172.21.15.135:0/1054627095 shutdown_connections 2024-04-17T06:10:03.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.424+0000 7fd077ed3640 1 -- 172.21.15.135:0/1054627095 >> 172.21.15.135:0/1054627095 conn(0x7fd0700b41a0 msgr2=0x7fd0700b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:03.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.424+0000 7fd077ed3640 1 -- 172.21.15.135:0/1054627095 shutdown_connections 2024-04-17T06:10:03.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.424+0000 7fd077ed3640 1 -- 172.21.15.135:0/1054627095 wait complete. 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.425+0000 7fd077ed3640 1 Processor -- start 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.425+0000 7fd077ed3640 1 -- start start 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd077ed3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c004d40 0x7fd07015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd077ed3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd07015f9b0 0x7fd07014a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd077ed3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07014cc60 0x7fd0700cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd077ed3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700d16a0 con 0x7fd07015f9b0 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd077ed3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700d10d0 con 0x7fd07014cc60 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd077ed3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd0700d1820 con 0x7fd05c004d40 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd07015f9b0 0x7fd07014a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd07015f9b0 0x7fd07014a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55292/0 (socket says 172.21.15.135:55292) 2024-04-17T06:10:03.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 -- 172.21.15.135:0/718574673 learned_addr learned my addr 172.21.15.135:0/718574673 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd076470640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07014cc60 0x7fd0700cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd075c6f640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c004d40 0x7fd07015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 -- 172.21.15.135:0/718574673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c004d40 msgr2=0x7fd07015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c004d40 0x7fd07015d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 -- 172.21.15.135:0/718574673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07014cc60 msgr2=0x7fd0700cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07014cc60 0x7fd0700cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 -- 172.21.15.135:0/718574673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd070160150 con 0x7fd07015f9b0 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd07546e640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd07015f9b0 0x7fd07014a9f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd060007340 tx=0x7fd060007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd0667fc640 1 -- 172.21.15.135:0/718574673 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06000f8e0 con 0x7fd07015f9b0 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd0667fc640 1 -- 172.21.15.135:0/718574673 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd060018070 con 0x7fd07015f9b0 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd0667fc640 1 -- 172.21.15.135:0/718574673 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06000fd00 con 0x7fd07015f9b0 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07006cbd0 con 0x7fd07015f9b0 2024-04-17T06:10:03.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.426+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07006d190 con 0x7fd07015f9b0 2024-04-17T06:10:03.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.427+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd070074940 con 0x7fd07015f9b0 2024-04-17T06:10:03.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.428+0000 7fd0667fc640 1 -- 172.21.15.135:0/718574673 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd06001c410 con 0x7fd07015f9b0 2024-04-17T06:10:03.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.428+0000 7fd0667fc640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd05406c920 0x7fd05406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.428+0000 7fd0667fc640 1 -- 172.21.15.135:0/718574673 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd06008ba00 con 0x7fd07015f9b0 2024-04-17T06:10:03.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.428+0000 7fd075c6f640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd05406c920 0x7fd05406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.429+0000 7fd075c6f640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd05406c920 0x7fd05406ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009c40 tx=0x7fd06c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.432+0000 7fd0667fc640 1 -- 172.21.15.135:0/718574673 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd060059ba0 con 0x7fd07015f9b0 2024-04-17T06:10:03.604 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.606+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd0700b3f80 con 0x7fd07015f9b0 2024-04-17T06:10:03.605 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.607+0000 7fd0667fc640 1 -- 172.21.15.135:0/718574673 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd0600594a0 con 0x7fd07015f9b0 2024-04-17T06:10:03.605 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:03.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.608+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd05406c920 msgr2=0x7fd05406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.608+0000 7fd077ed3640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd05406c920 0x7fd05406ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009c40 tx=0x7fd06c009190 comp rx=0 tx=0).stop 2024-04-17T06:10:03.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.608+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd07015f9b0 msgr2=0x7fd07014a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.608+0000 7fd077ed3640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd07015f9b0 0x7fd07014a9f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd060007340 tx=0x7fd060007740 comp rx=0 tx=0).stop 2024-04-17T06:10:03.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.608+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 shutdown_connections 2024-04-17T06:10:03.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.608+0000 7fd077ed3640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd07014cc60 0x7fd0700cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.608+0000 7fd077ed3640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd05406c920 0x7fd05406ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.607 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.608+0000 7fd077ed3640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd07015f9b0 0x7fd07014a9f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.609+0000 7fd077ed3640 1 --2- 172.21.15.135:0/718574673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd05c004d40 0x7fd07015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.609+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 >> 172.21.15.135:0/718574673 conn(0x7fd0700b41a0 msgr2=0x7fd0700cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:03.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.609+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 shutdown_connections 2024-04-17T06:10:03.608 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.609+0000 7fd077ed3640 1 -- 172.21.15.135:0/718574673 wait complete. 2024-04-17T06:10:03.618 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io' 2024-04-17T06:10:03.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.722+0000 7fcf437ba640 1 Processor -- start 2024-04-17T06:10:03.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.723+0000 7fcf437ba640 1 -- start start 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.723+0000 7fcf437ba640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcf3c15fbe0 0x7fcf3c0d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.723+0000 7fcf437ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c0d0d50 0x7fcf3c0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.723+0000 7fcf437ba640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf3c070e60 con 0x7fcf3c15f800 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf437ba640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3c06c8f0 con 0x7fcf3c15fbe0 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf437ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c06f070 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf41d57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c0d0d50 0x7fcf3c0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf41d57640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c0d0d50 0x7fcf3c0cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:36830/0 (socket says 172.21.15.135:36830) 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf41d57640 1 -- 172.21.15.135:0/2057610492 learned_addr learned my addr 172.21.15.135:0/2057610492 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:03.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf40d55640 1 --2- 172.21.15.135:0/2057610492 >> v2:172.21.15.190:3300/0 conn(0x7fcf3c15fbe0 0x7fcf3c0d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf41d57640 1 -- 172.21.15.135:0/2057610492 >> v2:172.21.15.190:3300/0 conn(0x7fcf3c15fbe0 msgr2=0x7fcf3c0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:03.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf41d57640 1 --2- 172.21.15.135:0/2057610492 >> v2:172.21.15.190:3300/0 conn(0x7fcf3c15fbe0 0x7fcf3c0d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf41d57640 1 -- 172.21.15.135:0/2057610492 >> v1:172.21.15.190:6789/0 conn(0x7fcf3c15f800 legacy=0x7fcf3c070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf41d57640 1 -- 172.21.15.135:0/2057610492 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c06bb30 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.724+0000 7fcf40d55640 1 --2- 172.21.15.135:0/2057610492 >> v2:172.21.15.190:3300/0 conn(0x7fcf3c15fbe0 0x7fcf3c0d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:10:03.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.725+0000 7fcf41d57640 1 --2- 172.21.15.135:0/2057610492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c0d0d50 0x7fcf3c0cde00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009e50 tx=0x7fcf3402bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a05ec7e6dbd7130 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.725+0000 7fcf2ffff640 1 -- 172.21.15.135:0/2057610492 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3402cb90 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.725+0000 7fcf2ffff640 1 -- 172.21.15.135:0/2057610492 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3402cd30 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.725+0000 7fcf2ffff640 1 -- 172.21.15.135:0/2057610492 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34034610 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.726+0000 7fcf437ba640 1 -- 172.21.15.135:0/2057610492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c0d0d50 msgr2=0x7fcf3c0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.726+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2057610492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c0d0d50 0x7fcf3c0cde00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009e50 tx=0x7fcf3402bff0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.726+0000 7fcf437ba640 1 -- 172.21.15.135:0/2057610492 shutdown_connections 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.726+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2057610492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c0d0d50 0x7fcf3c0cde00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.726+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2057610492 >> v2:172.21.15.190:3300/0 conn(0x7fcf3c15fbe0 0x7fcf3c0d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.726+0000 7fcf437ba640 1 -- 172.21.15.135:0/2057610492 >> 172.21.15.135:0/2057610492 conn(0x7fcf3c09a9f0 msgr2=0x7fcf3c0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.726+0000 7fcf437ba640 1 -- 172.21.15.135:0/2057610492 shutdown_connections 2024-04-17T06:10:03.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.726+0000 7fcf437ba640 1 -- 172.21.15.135:0/2057610492 wait complete. 2024-04-17T06:10:03.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.727+0000 7fcf437ba640 1 Processor -- start 2024-04-17T06:10:03.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.727+0000 7fcf437ba640 1 -- start start 2024-04-17T06:10:03.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf437ba640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf3c0d0d50 0x7fcf3c14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf437ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c15f800 0x7fcf3c0b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf437ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c15fbe0 0x7fcf3c1f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf437ba640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c06c0f0 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf437ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c06c460 con 0x7fcf3c15f800 2024-04-17T06:10:03.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf437ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c0ce340 con 0x7fcf3c15fbe0 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41556640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf3c0d0d50 0x7fcf3c14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41556640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf3c0d0d50 0x7fcf3c14fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55308/0 (socket says 172.21.15.135:55308) 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41556640 1 -- 172.21.15.135:0/2680768685 learned_addr learned my addr 172.21.15.135:0/2680768685 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41d57640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c15fbe0 0x7fcf3c1f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf40d55640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c15f800 0x7fcf3c0b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41556640 1 -- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c15fbe0 msgr2=0x7fcf3c1f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41556640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c15fbe0 0x7fcf3c1f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41556640 1 -- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c15f800 msgr2=0x7fcf3c0b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41556640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c15f800 0x7fcf3c0b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf41556640 1 -- 172.21.15.135:0/2680768685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c0b4fc0 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.728+0000 7fcf40d55640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c15f800 0x7fcf3c0b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.729+0000 7fcf41556640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf3c0d0d50 0x7fcf3c14fb90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c430 tx=0x7fcf3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.729+0000 7fcf2dffb640 1 -- 172.21.15.135:0/2680768685 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf380052d0 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.729+0000 7fcf2dffb640 1 -- 172.21.15.135:0/2680768685 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf38005490 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.729+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c1f8de0 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.729+0000 7fcf2dffb640 1 -- 172.21.15.135:0/2680768685 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38007260 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.729+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c160280 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.730+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3c074940 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.730+0000 7fcf2dffb640 1 -- 172.21.15.135:0/2680768685 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf38007400 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.731+0000 7fcf2dffb640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcf2006c8f0 0x7fcf2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:03.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.731+0000 7fcf2dffb640 1 -- 172.21.15.135:0/2680768685 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fcf3808b360 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.731+0000 7fcf40d55640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcf2006c8f0 0x7fcf2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:03.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.731+0000 7fcf40d55640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcf2006c8f0 0x7fcf2006edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0cdaf0 tx=0x7fcf30008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:03.733 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.735+0000 7fcf2dffb640 1 -- 172.21.15.135:0/2680768685 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf38058dd0 con 0x7fcf3c0d0d50 2024-04-17T06:10:03.974 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:03.976+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io"]} v 0) -- 0x7fcf3c073be0 con 0x7fcf3c0d0d50 2024-04-17T06:10:04.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.029+0000 7fcf2dffb640 1 -- 172.21.15.135:0/2680768685 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_admin.TestLabeledPerfCounters.test_per_client_labeled_perf_counters_io"]}]=0 v65) ==== 170+0+0 (secure 0 0 0) 0x7fcf3805d0d0 con 0x7fcf3c0d0d50 2024-04-17T06:10:04.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcf2006c8f0 msgr2=0x7fcf2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcf2006c8f0 0x7fcf2006edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0cdaf0 tx=0x7fcf30008040 comp rx=0 tx=0).stop 2024-04-17T06:10:04.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf3c0d0d50 msgr2=0x7fcf3c14fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf3c0d0d50 0x7fcf3c14fb90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c430 tx=0x7fcf3800c8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 shutdown_connections 2024-04-17T06:10:04.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcf3c15fbe0 0x7fcf3c1f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcf3c15f800 0x7fcf3c0b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcf2006c8f0 0x7fcf2006edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 --2- 172.21.15.135:0/2680768685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf3c0d0d50 0x7fcf3c14fb90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 >> 172.21.15.135:0/2680768685 conn(0x7fcf3c09a9f0 msgr2=0x7fcf3c15e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:04.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 shutdown_connections 2024-04-17T06:10:04.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.031+0000 7fcf437ba640 1 -- 172.21.15.135:0/2680768685 wait complete. 2024-04-17T06:10:04.040 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-17T06:10:04.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.148+0000 7fea84035640 1 Processor -- start 2024-04-17T06:10:04.147 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.149+0000 7fea84035640 1 -- start start 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.149+0000 7fea84035640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c15bb00 0x7fea7c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea84035640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fea7c0b3590 0x7fea7c0af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea84035640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fea7c0afc00 0x7fea7c071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea84035640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c0b0060 con 0x7fea7c0b3590 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea84035640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c0b4660 con 0x7fea7c0afc00 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea84035640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c0b3a50 con 0x7fea7c15bb00 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea815d0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fea7c0b3590 0x7fea7c0af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea815d0640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fea7c0b3590 0x7fea7c0af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55322/0 (socket says 172.21.15.135:55322) 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea815d0640 1 -- 172.21.15.135:0/811423584 learned_addr learned my addr 172.21.15.135:0/811423584 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:04.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea81dd1640 1 --2- 172.21.15.135:0/811423584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c15bb00 0x7fea7c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea825d2640 1 --2- 172.21.15.135:0/811423584 >> v2:172.21.15.190:3300/0 conn(0x7fea7c0afc00 0x7fea7c071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea815d0640 1 -- 172.21.15.135:0/811423584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c15bb00 msgr2=0x7fea7c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea815d0640 1 --2- 172.21.15.135:0/811423584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c15bb00 0x7fea7c0b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea815d0640 1 -- 172.21.15.135:0/811423584 >> v2:172.21.15.190:3300/0 conn(0x7fea7c0afc00 msgr2=0x7fea7c071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea815d0640 1 --2- 172.21.15.135:0/811423584 >> v2:172.21.15.190:3300/0 conn(0x7fea7c0afc00 0x7fea7c071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.150+0000 7fea815d0640 1 -- 172.21.15.135:0/811423584 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c0b3d30 con 0x7fea7c0b3590 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.151+0000 7fea815d0640 1 --2- 172.21.15.135:0/811423584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c0b3590 0x7fea7c0af6c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0ccdd0 tx=0x7fea7002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc93c3595fd7701 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.151+0000 7fea80dcf640 1 -- 172.21.15.135:0/811423584 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7003b070 con 0x7fea7c0b3590 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.151+0000 7fea80dcf640 1 -- 172.21.15.135:0/811423584 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea70036440 con 0x7fea7c0b3590 2024-04-17T06:10:04.150 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.151+0000 7fea80dcf640 1 -- 172.21.15.135:0/811423584 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70036810 con 0x7fea7c0b3590 2024-04-17T06:10:04.153 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.151+0000 7fea84035640 1 -- 172.21.15.135:0/811423584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c0b3590 msgr2=0x7fea7c0af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.151+0000 7fea84035640 1 --2- 172.21.15.135:0/811423584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c0b3590 0x7fea7c0af6c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0ccdd0 tx=0x7fea7002e0e0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.152+0000 7fea84035640 1 -- 172.21.15.135:0/811423584 shutdown_connections 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.152+0000 7fea84035640 1 --2- 172.21.15.135:0/811423584 >> v2:172.21.15.190:3300/0 conn(0x7fea7c0afc00 0x7fea7c071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.152+0000 7fea84035640 1 --2- 172.21.15.135:0/811423584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c0b3590 0x7fea7c0af6c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.152+0000 7fea84035640 1 --2- 172.21.15.135:0/811423584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c15bb00 0x7fea7c0b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.152+0000 7fea84035640 1 -- 172.21.15.135:0/811423584 >> 172.21.15.135:0/811423584 conn(0x7fea7c0d0520 msgr2=0x7fea7c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.152+0000 7fea84035640 1 -- 172.21.15.135:0/811423584 shutdown_connections 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.152+0000 7fea84035640 1 -- 172.21.15.135:0/811423584 wait complete. 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.153+0000 7fea84035640 1 Processor -- start 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.153+0000 7fea84035640 1 -- start start 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.153+0000 7fea84035640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c0afc00 0x7fea7c14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.153+0000 7fea84035640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea7c0b3590 0x7fea7c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.154 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.153+0000 7fea84035640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c15bb00 0x7fea7c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.153+0000 7fea84035640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0b5690 con 0x7fea7c15bb00 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea84035640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0b42f0 con 0x7fea7c0b3590 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea84035640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c0b5810 con 0x7fea7c0afc00 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c15bb00 0x7fea7c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c15bb00 0x7fea7c15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55334/0 (socket says 172.21.15.135:55334) 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 -- 172.21.15.135:0/2772162543 learned_addr learned my addr 172.21.15.135:0/2772162543 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea81dd1640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c0afc00 0x7fea7c14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea815d0640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea7c0b3590 0x7fea7c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 -- 172.21.15.135:0/2772162543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c0afc00 msgr2=0x7fea7c14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c0afc00 0x7fea7c14a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 -- 172.21.15.135:0/2772162543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea7c0b3590 msgr2=0x7fea7c15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea7c0b3590 0x7fea7c15e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.155 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 -- 172.21.15.135:0/2772162543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c0ae3a0 con 0x7fea7c15bb00 2024-04-17T06:10:04.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea825d2640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c15bb00 0x7fea7c15c880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fea7400ef00 tx=0x7fea7400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:04.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea6e7fc640 1 -- 172.21.15.135:0/2772162543 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74018020 con 0x7fea7c15bb00 2024-04-17T06:10:04.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea6e7fc640 1 -- 172.21.15.135:0/2772162543 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea74014400 con 0x7fea7c15bb00 2024-04-17T06:10:04.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea6e7fc640 1 -- 172.21.15.135:0/2772162543 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea740147f0 con 0x7fea7c15bb00 2024-04-17T06:10:04.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c0ae680 con 0x7fea7c15bb00 2024-04-17T06:10:04.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.154+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c0aec40 con 0x7fea7c15bb00 2024-04-17T06:10:04.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.155+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c074940 con 0x7fea7c15bb00 2024-04-17T06:10:04.156 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.156+0000 7fea6e7fc640 1 -- 172.21.15.135:0/2772162543 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea74005d10 con 0x7fea7c15bb00 2024-04-17T06:10:04.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.156+0000 7fea6e7fc640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fea6006c8f0 0x7fea6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.156+0000 7fea6e7fc640 1 -- 172.21.15.135:0/2772162543 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fea7408c430 con 0x7fea7c15bb00 2024-04-17T06:10:04.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.156+0000 7fea81dd1640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fea6006c8f0 0x7fea6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.157+0000 7fea81dd1640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fea6006c8f0 0x7fea6006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea78009aa0 tx=0x7fea78009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:04.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.161+0000 7fea6e7fc640 1 -- 172.21.15.135:0/2772162543 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea74022070 con 0x7fea7c15bb00 2024-04-17T06:10:04.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.336+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fea7c06fd10 con 0x7fea7c15bb00 2024-04-17T06:10:04.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.339+0000 7fea6e7fc640 1 -- 172.21.15.135:0/2772162543 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fea74059ea0 con 0x7fea7c15bb00 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fea6006c8f0 msgr2=0x7fea6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fea6006c8f0 0x7fea6006edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea78009aa0 tx=0x7fea78009190 comp rx=0 tx=0).stop 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c15bb00 msgr2=0x7fea7c15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c15bb00 0x7fea7c15c880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fea7400ef00 tx=0x7fea7400b480 comp rx=0 tx=0).stop 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 shutdown_connections 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fea6006c8f0 0x7fea6006edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea7c15bb00 0x7fea7c15c880 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea7c0b3590 0x7fea7c15e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 --2- 172.21.15.135:0/2772162543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea7c0afc00 0x7fea7c14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 >> 172.21.15.135:0/2772162543 conn(0x7fea7c0d0520 msgr2=0x7fea7c157ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.341+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 shutdown_connections 2024-04-17T06:10:04.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.342+0000 7fea84035640 1 -- 172.21.15.135:0/2772162543 wait complete. 2024-04-17T06:10:04.351 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-17T06:10:04.351 DEBUG:teuthology.orchestra.run.smithi135:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T06:10:04.393 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-17T06:10:04.445 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-17T06:10:04.445 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:04.445 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:04.461 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:04.462 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:04.517 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:04.518 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:04.573 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:04.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.720+0000 7ff498500640 1 Processor -- start 2024-04-17T06:10:04.768 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.720+0000 7ff498500640 1 -- start start 2024-04-17T06:10:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff498500640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff49015fbe0 0x7ff4900d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff498500640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4900d0d50 0x7ff4900cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff498500640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff490070e60 con 0x7ff49015f800 2024-04-17T06:10:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff498500640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff49006c8f0 con 0x7ff49015fbe0 2024-04-17T06:10:04.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff498500640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff49006f070 con 0x7ff4900d0d50 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff495a9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff49015fbe0 0x7ff4900d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff495a9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff49015fbe0 0x7ff4900d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55630/0 (socket says 172.21.15.135:55630) 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff495a9b640 1 -- 172.21.15.135:0/871315591 learned_addr learned my addr 172.21.15.135:0/871315591 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff495a9b640 1 -- 172.21.15.135:0/871315591 >> v2:172.21.15.190:3300/0 conn(0x7ff4900d0d50 msgr2=0x7ff4900cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff495a9b640 1 --2- 172.21.15.135:0/871315591 >> v2:172.21.15.190:3300/0 conn(0x7ff4900d0d50 0x7ff4900cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff495a9b640 1 -- 172.21.15.135:0/871315591 >> v1:172.21.15.190:6789/0 conn(0x7ff49015f800 legacy=0x7ff490070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.721+0000 7ff495a9b640 1 -- 172.21.15.135:0/871315591 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49006bb30 con 0x7ff49015fbe0 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.722+0000 7ff495a9b640 1 --2- 172.21.15.135:0/871315591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4900d0810 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff49009afa0 tx=0x7ff48002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534a3ff312313146 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:04.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.722+0000 7ff49529a640 1 -- 172.21.15.135:0/871315591 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480034070 con 0x7ff49015fbe0 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.722+0000 7ff49529a640 1 -- 172.21.15.135:0/871315591 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff480030860 con 0x7ff49015fbe0 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.722+0000 7ff49529a640 1 -- 172.21.15.135:0/871315591 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480030c30 con 0x7ff49015fbe0 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.723+0000 7ff498500640 1 -- 172.21.15.135:0/871315591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 msgr2=0x7ff4900d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.723+0000 7ff498500640 1 --2- 172.21.15.135:0/871315591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4900d0810 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff49009afa0 tx=0x7ff48002b9a0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.723+0000 7ff498500640 1 -- 172.21.15.135:0/871315591 shutdown_connections 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.723+0000 7ff498500640 1 --2- 172.21.15.135:0/871315591 >> v2:172.21.15.190:3300/0 conn(0x7ff4900d0d50 0x7ff4900cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.723+0000 7ff498500640 1 --2- 172.21.15.135:0/871315591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4900d0810 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.723+0000 7ff498500640 1 -- 172.21.15.135:0/871315591 >> 172.21.15.135:0/871315591 conn(0x7ff49009a9f0 msgr2=0x7ff4900b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.723+0000 7ff498500640 1 -- 172.21.15.135:0/871315591 shutdown_connections 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.723+0000 7ff498500640 1 -- 172.21.15.135:0/871315591 wait complete. 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.724+0000 7ff498500640 1 Processor -- start 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.724+0000 7ff498500640 1 -- start start 2024-04-17T06:10:04.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff498500640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900d0d50 0x7ff490152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff498500640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49015f800 0x7ff490161e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff498500640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4901580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff498500640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff49006c0f0 con 0x7ff49015fbe0 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff498500640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff49006c460 con 0x7ff4900d0d50 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff498500640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff4900ce340 con 0x7ff49015f800 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4901580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4901580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55632/0 (socket says 172.21.15.135:55632) 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 -- 172.21.15.135:0/2872213983 learned_addr learned my addr 172.21.15.135:0/2872213983 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff495a9b640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49015f800 0x7ff490161e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff49629c640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900d0d50 0x7ff490152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 -- 172.21.15.135:0/2872213983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49015f800 msgr2=0x7ff490161e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49015f800 0x7ff490161e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 -- 172.21.15.135:0/2872213983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900d0d50 msgr2=0x7ff490152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.778 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900d0d50 0x7ff490152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 -- 172.21.15.135:0/2872213983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4900c5110 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.725+0000 7ff496a9d640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4901580a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b970 tx=0x7ff48800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.726+0000 7ff486ffd640 1 -- 172.21.15.135:0/2872213983 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488019070 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.726+0000 7ff486ffd640 1 -- 172.21.15.135:0/2872213983 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48800cb60 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.726+0000 7ff486ffd640 1 -- 172.21.15.135:0/2872213983 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488014440 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.726+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4900c53f0 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.726+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4900c59b0 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.727+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49015e120 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.727+0000 7ff486ffd640 1 -- 172.21.15.135:0/2872213983 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff48801c070 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.727+0000 7ff486ffd640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.727+0000 7ff486ffd640 1 -- 172.21.15.135:0/2872213983 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff48808bc70 con 0x7ff49015fbe0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.727+0000 7ff49629c640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.728+0000 7ff49629c640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007fd0 tx=0x7ff48c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:04.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.732+0000 7ff486ffd640 1 -- 172.21.15.135:0/2872213983 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff49015e120 con 0x7ff49015fbe0 2024-04-17T06:10:04.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.908+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff490074940 con 0x7ff49015fbe0 2024-04-17T06:10:04.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.909+0000 7ff486ffd640 1 -- 172.21.15.135:0/2872213983 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18950 (secure 0 0 0) 0x7ff488059ca0 con 0x7ff49015fbe0 2024-04-17T06:10:04.907 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:04.908 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:01.008924+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:04.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47406c8f0 msgr2=0x7ff47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47406c8f0 0x7ff47406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007fd0 tx=0x7ff48c009040 comp rx=0 tx=0).stop 2024-04-17T06:10:04.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 msgr2=0x7ff4901580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:04.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4901580a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b970 tx=0x7ff48800be30 comp rx=0 tx=0).stop 2024-04-17T06:10:04.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 shutdown_connections 2024-04-17T06:10:04.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff47406c8f0 0x7ff47406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff49015fbe0 0x7ff4901580a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff49015f800 0x7ff490161e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 --2- 172.21.15.135:0/2872213983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4900d0d50 0x7ff490152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:04.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 >> 172.21.15.135:0/2872213983 conn(0x7ff49009a9f0 msgr2=0x7ff49015caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:04.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 shutdown_connections 2024-04-17T06:10:04.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:04.911+0000 7ff498500640 1 -- 172.21.15.135:0/2872213983 wait complete. 2024-04-17T06:10:04.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:10:04.921 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:10:05.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.031+0000 7f9463a2c640 1 Processor -- start 2024-04-17T06:10:05.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.031+0000 7f9463a2c640 1 -- start start 2024-04-17T06:10:05.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9463a2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c166bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9463a2c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945c0cd0a0 con 0x7f945c06f9d0 2024-04-17T06:10:05.030 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9463a2c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f945c0cd380 con 0x7f945c15b940 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9463a2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f945c06f1d0 con 0x7f945c1647c0 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9460fc7640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f945c06f9d0 0x7f945c06fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42272/0 (socket says 172.21.15.135:42272) 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9460fc7640 1 -- 172.21.15.135:0/128309977 learned_addr learned my addr 172.21.15.135:0/128309977 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9461fc9640 1 --2- 172.21.15.135:0/128309977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c166bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9453fff640 1 -- 172.21.15.135:0/128309977 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2521866824 0 0) 0x7f945c0cd0a0 con 0x7f945c06f9d0 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9453fff640 1 -- 172.21.15.135:0/128309977 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9448003520 con 0x7f945c06f9d0 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9461fc9640 1 -- 172.21.15.135:0/128309977 >> v1:172.21.15.190:6789/0 conn(0x7f945c15b940 legacy=0x7f945c0cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9461fc9640 1 -- 172.21.15.135:0/128309977 >> v1:172.21.15.135:6789/0 conn(0x7f945c06f9d0 legacy=0x7f945c06fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.031 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.032+0000 7f9461fc9640 1 -- 172.21.15.135:0/128309977 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c06f4b0 con 0x7f945c1647c0 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.033+0000 7f9461fc9640 1 --2- 172.21.15.135:0/128309977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c166bb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9454009dc0 tx=0x7f945402bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a7b9509e5114eb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.033+0000 7f9453fff640 1 -- 172.21.15.135:0/128309977 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454039070 con 0x7f945c1647c0 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.033+0000 7f9453fff640 1 -- 172.21.15.135:0/128309977 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f945402cd50 con 0x7f945c1647c0 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.033+0000 7f9453fff640 1 -- 172.21.15.135:0/128309977 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454034640 con 0x7f945c1647c0 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.034+0000 7f9463a2c640 1 -- 172.21.15.135:0/128309977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 msgr2=0x7f945c166bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.034+0000 7f9463a2c640 1 --2- 172.21.15.135:0/128309977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c166bb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9454009dc0 tx=0x7f945402bfa0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.034+0000 7f9463a2c640 1 -- 172.21.15.135:0/128309977 shutdown_connections 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.034+0000 7f9463a2c640 1 --2- 172.21.15.135:0/128309977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c166bb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.034+0000 7f9463a2c640 1 -- 172.21.15.135:0/128309977 >> 172.21.15.135:0/128309977 conn(0x7f945c09a9f0 msgr2=0x7f945c0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:05.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.034+0000 7f9463a2c640 1 -- 172.21.15.135:0/128309977 shutdown_connections 2024-04-17T06:10:05.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.034+0000 7f9463a2c640 1 -- 172.21.15.135:0/128309977 wait complete. 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.035+0000 7f9463a2c640 1 Processor -- start 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.035+0000 7f9463a2c640 1 -- start start 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9463a2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f945c06f9d0 0x7f945c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9463a2c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945c15b940 0x7f945c1f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9463a2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9463a2c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f945c0ccc90 con 0x7f945c15b940 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9463a2c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f945c15bd20 con 0x7f945c06f9d0 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9463a2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f945c0cce10 con 0x7f945c1647c0 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9460fc7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945c15b940 0x7f945c1f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9460fc7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945c15b940 0x7f945c1f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55646/0 (socket says 172.21.15.135:55646) 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9460fc7640 1 -- 172.21.15.135:0/1961027311 learned_addr learned my addr 172.21.15.135:0/1961027311 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f9461fc9640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.036+0000 7f94617c8640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f945c06f9d0 0x7f945c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9460fc7640 1 -- 172.21.15.135:0/1961027311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 msgr2=0x7f945c1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9460fc7640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9460fc7640 1 -- 172.21.15.135:0/1961027311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f945c06f9d0 msgr2=0x7f945c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9460fc7640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f945c06f9d0 0x7f945c15c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9460fc7640 1 -- 172.21.15.135:0/1961027311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c15ce50 con 0x7f945c15b940 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9460fc7640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945c15b940 0x7f945c1f84a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c4d0 tx=0x7f944c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9451ffb640 1 -- 172.21.15.135:0/1961027311 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c017070 con 0x7f945c15b940 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9451ffb640 1 -- 172.21.15.135:0/1961027311 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f944c005ce0 con 0x7f945c15b940 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9451ffb640 1 -- 172.21.15.135:0/1961027311 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c007270 con 0x7f945c15b940 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c15dad0 con 0x7f945c15b940 2024-04-17T06:10:05.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.037+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c15e090 con 0x7f945c15b940 2024-04-17T06:10:05.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.038+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945c1f7d70 con 0x7f945c15b940 2024-04-17T06:10:05.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.039+0000 7f9451ffb640 1 -- 172.21.15.135:0/1961027311 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f944c01a070 con 0x7f945c15b940 2024-04-17T06:10:05.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.039+0000 7f9451ffb640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f944006c8f0 0x7f944006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.039+0000 7f9451ffb640 1 -- 172.21.15.135:0/1961027311 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f944c08b180 con 0x7f945c15b940 2024-04-17T06:10:05.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.039+0000 7f94617c8640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f944006c8f0 0x7f944006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.040+0000 7f94617c8640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f944006c8f0 0x7f944006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f945c0c54b0 tx=0x7f9458009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.044+0000 7f9451ffb640 1 -- 172.21.15.135:0/1961027311 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f944c058b60 con 0x7f945c15b940 2024-04-17T06:10:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.216+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f945c074940 con 0x7f945c15b940 2024-04-17T06:10:05.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.216+0000 7f9451ffb640 1 -- 172.21.15.135:0/1961027311 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f944c05ce60 con 0x7f945c15b940 2024-04-17T06:10:05.215 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f944006c8f0 msgr2=0x7f944006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f944006c8f0 0x7f944006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f945c0c54b0 tx=0x7f9458009040 comp rx=0 tx=0).stop 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945c15b940 msgr2=0x7f945c1f84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945c15b940 0x7f945c1f84a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c4d0 tx=0x7f944c00c990 comp rx=0 tx=0).stop 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 shutdown_connections 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f945c1647c0 0x7f945c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f944006c8f0 0x7f944006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f945c15b940 0x7f945c1f84a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 --2- 172.21.15.135:0/1961027311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f945c06f9d0 0x7f945c15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.218+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 >> 172.21.15.135:0/1961027311 conn(0x7f945c09a9f0 msgr2=0x7f945c15ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:05.217 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.219+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 shutdown_connections 2024-04-17T06:10:05.218 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.219+0000 7f9463a2c640 1 -- 172.21.15.135:0/1961027311 wait complete. 2024-04-17T06:10:05.228 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:10:05.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.336+0000 7f7324a93640 1 Processor -- start 2024-04-17T06:10:05.334 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.336+0000 7f7324a93640 1 -- start start 2024-04-17T06:10:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f7324a93640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f732015c8a0 0x7f73200b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f7324a93640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f73200cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f7324a93640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73200cd5e0 0x7f73200cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f7324a93640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f732006c600 con 0x7f73200cd5e0 2024-04-17T06:10:05.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f7324a93640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f732006ba30 con 0x7f732015c8a0 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f7324a93640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f732006bcd0 con 0x7f73200b3d40 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73200cd5e0 0x7f73200cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731ed9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f73200cd5e0 0x7f73200cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55652/0 (socket says 172.21.15.135:55652) 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731ed9d640 1 -- 172.21.15.135:0/2075398044 learned_addr learned my addr 172.21.15.135:0/2075398044 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731dd9b640 1 --2- 172.21.15.135:0/2075398044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f73200cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731e59c640 1 --2- 172.21.15.135:0/2075398044 >> v2:172.21.15.190:3300/0 conn(0x7f732015c8a0 0x7f73200b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731ed9d640 1 -- 172.21.15.135:0/2075398044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 msgr2=0x7f73200cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731ed9d640 1 --2- 172.21.15.135:0/2075398044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f73200cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731ed9d640 1 -- 172.21.15.135:0/2075398044 >> v2:172.21.15.190:3300/0 conn(0x7f732015c8a0 msgr2=0x7f73200b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731ed9d640 1 --2- 172.21.15.135:0/2075398044 >> v2:172.21.15.190:3300/0 conn(0x7f732015c8a0 0x7f73200b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731ed9d640 1 -- 172.21.15.135:0/2075398044 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732006bfb0 con 0x7f73200cd5e0 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.337+0000 7f731e59c640 1 --2- 172.21.15.135:0/2075398044 >> v2:172.21.15.190:3300/0 conn(0x7f732015c8a0 0x7f73200b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:05.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.338+0000 7f731ed9d640 1 --2- 172.21.15.135:0/2075398044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200cf150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f73200cdf60 tx=0x7f731802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4ad1075498ab03 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.338+0000 7f731d59a640 1 -- 172.21.15.135:0/2075398044 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318035020 con 0x7f73200cd5e0 2024-04-17T06:10:05.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.338+0000 7f731d59a640 1 -- 172.21.15.135:0/2075398044 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7318031870 con 0x7f73200cd5e0 2024-04-17T06:10:05.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.338+0000 7f731d59a640 1 -- 172.21.15.135:0/2075398044 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318031c40 con 0x7f73200cd5e0 2024-04-17T06:10:05.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 -- 172.21.15.135:0/2075398044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 msgr2=0x7f73200cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 --2- 172.21.15.135:0/2075398044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200cf150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f73200cdf60 tx=0x7f731802b910 comp rx=0 tx=0).stop 2024-04-17T06:10:05.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 -- 172.21.15.135:0/2075398044 shutdown_connections 2024-04-17T06:10:05.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 --2- 172.21.15.135:0/2075398044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200cf150 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 --2- 172.21.15.135:0/2075398044 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f73200cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 --2- 172.21.15.135:0/2075398044 >> v2:172.21.15.190:3300/0 conn(0x7f732015c8a0 0x7f73200b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 -- 172.21.15.135:0/2075398044 >> 172.21.15.135:0/2075398044 conn(0x7f732009a9f0 msgr2=0x7f73200b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:05.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 -- 172.21.15.135:0/2075398044 shutdown_connections 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.339+0000 7f7324a93640 1 -- 172.21.15.135:0/2075398044 wait complete. 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.340+0000 7f7324a93640 1 Processor -- start 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.340+0000 7f7324a93640 1 -- start start 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f7324a93640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f732015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f7324a93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f7324a93640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732015c8a0 0x7f73200ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f7324a93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73200ced60 con 0x7f73200cd5e0 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f7324a93640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f73200cda70 con 0x7f732015c8a0 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f7324a93640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f73200d0370 con 0x7f73200b3d40 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55660/0 (socket says 172.21.15.135:55660) 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731dd9b640 1 -- 172.21.15.135:0/4238613680 learned_addr learned my addr 172.21.15.135:0/4238613680 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:05.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731ed9d640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732015c8a0 0x7f73200ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731e59c640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f732015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731dd9b640 1 -- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 msgr2=0x7f732015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731dd9b640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f732015ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731dd9b640 1 -- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732015c8a0 msgr2=0x7f73200ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731dd9b640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732015c8a0 0x7f73200ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731dd9b640 1 -- 172.21.15.135:0/4238613680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200b4650 con 0x7f73200cd5e0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.341+0000 7f731e59c640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f732015ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.342+0000 7f731dd9b640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200c8650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f731400e890 tx=0x7f731400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.342+0000 7f730effd640 1 -- 172.21.15.135:0/4238613680 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314017070 con 0x7f73200cd5e0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.342+0000 7f730effd640 1 -- 172.21.15.135:0/4238613680 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731400c940 con 0x7f73200cd5e0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.342+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73200b4930 con 0x7f73200cd5e0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.342+0000 7f730effd640 1 -- 172.21.15.135:0/4238613680 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731400cd10 con 0x7f73200cd5e0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.342+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73200b4ef0 con 0x7f73200cd5e0 2024-04-17T06:10:05.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.343+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7320074940 con 0x7f73200cd5e0 2024-04-17T06:10:05.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.343+0000 7f730effd640 1 -- 172.21.15.135:0/4238613680 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f731401a070 con 0x7f73200cd5e0 2024-04-17T06:10:05.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.343+0000 7f730effd640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.344+0000 7f730effd640 1 -- 172.21.15.135:0/4238613680 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f731408c280 con 0x7f73200cd5e0 2024-04-17T06:10:05.344 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.344+0000 7f731e59c640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.344+0000 7f731e59c640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7310009540 tx=0x7f731000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.348+0000 7f730effd640 1 -- 172.21.15.135:0/4238613680 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f731405a2c0 con 0x7f73200cd5e0 2024-04-17T06:10:05.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.523+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f73200af2d0 con 0x7f73200cd5e0 2024-04-17T06:10:05.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.524+0000 7f730effd640 1 -- 172.21.15.135:0/4238613680 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7314059bc0 con 0x7f73200cd5e0 2024-04-17T06:10:05.522 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.525+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 msgr2=0x7f730006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.525+0000 7f7324a93640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7310009540 tx=0x7f731000b040 comp rx=0 tx=0).stop 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.525+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 msgr2=0x7f73200c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.525+0000 7f7324a93640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200c8650 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f731400e890 tx=0x7f731400ed50 comp rx=0 tx=0).stop 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.526+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 shutdown_connections 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.526+0000 7f7324a93640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732015c8a0 0x7f73200ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.526+0000 7f7324a93640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f730006c8f0 0x7f730006edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.526+0000 7f7324a93640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73200cd5e0 0x7f73200c8650 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.526+0000 7f7324a93640 1 --2- 172.21.15.135:0/4238613680 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73200b3d40 0x7f732015ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.526+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 >> 172.21.15.135:0/4238613680 conn(0x7f732009a9f0 msgr2=0x7f73200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:05.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.526+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 shutdown_connections 2024-04-17T06:10:05.525 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.526+0000 7f7324a93640 1 -- 172.21.15.135:0/4238613680 wait complete. 2024-04-17T06:10:05.535 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:10:05.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.643+0000 7f45794d9640 1 Processor -- start 2024-04-17T06:10:05.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.643+0000 7f45794d9640 1 -- start start 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f45794d9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f457415ee30 0x7f457409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f45794d9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409bbe0 0x7f457409af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f45794d9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457406db80 con 0x7f457409b470 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f45794d9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f457406de60 con 0x7f457415ee30 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f45794d9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f45740cce30 con 0x7f457409bbe0 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f4572ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f457415ee30 0x7f457409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f45737fe640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f457409b470 0x7f45740d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42284/0 (socket says 172.21.15.135:42284) 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f45737fe640 1 -- 172.21.15.135:0/2856672779 learned_addr learned my addr 172.21.15.135:0/2856672779 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.644+0000 7f45727fc640 1 --2- 172.21.15.135:0/2856672779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409bbe0 0x7f457409af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.645+0000 7f4572ffd640 1 -- 172.21.15.135:0/2856672779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409bbe0 msgr2=0x7f457409af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.645+0000 7f4572ffd640 1 --2- 172.21.15.135:0/2856672779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409bbe0 0x7f457409af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.643 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.645+0000 7f4572ffd640 1 -- 172.21.15.135:0/2856672779 >> v1:172.21.15.135:6789/0 conn(0x7f457409b470 legacy=0x7f45740d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.645+0000 7f4572ffd640 1 -- 172.21.15.135:0/2856672779 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45740cd110 con 0x7f457415ee30 2024-04-17T06:10:05.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.645+0000 7f4571ffb640 1 -- 172.21.15.135:0/2856672779 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845155404 0 0) 0x7f457406db80 con 0x7f457409b470 2024-04-17T06:10:05.645 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.645+0000 7f4572ffd640 1 --2- 172.21.15.135:0/2856672779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f457409a9f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4564009d90 tx=0x7f456402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbf553c77ef263 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.645+0000 7f4571ffb640 1 -- 172.21.15.135:0/2856672779 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564039070 con 0x7f457415ee30 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f4571ffb640 1 -- 172.21.15.135:0/2856672779 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456402cc50 con 0x7f457415ee30 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f4571ffb640 1 -- 172.21.15.135:0/2856672779 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564034440 con 0x7f457415ee30 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f45794d9640 1 -- 172.21.15.135:0/2856672779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 msgr2=0x7f457409a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f45794d9640 1 --2- 172.21.15.135:0/2856672779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f457409a9f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4564009d90 tx=0x7f456402bea0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f45794d9640 1 -- 172.21.15.135:0/2856672779 shutdown_connections 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f45794d9640 1 --2- 172.21.15.135:0/2856672779 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409bbe0 0x7f457409af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f45794d9640 1 --2- 172.21.15.135:0/2856672779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f457409a9f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f45794d9640 1 -- 172.21.15.135:0/2856672779 >> 172.21.15.135:0/2856672779 conn(0x7f45740b41a0 msgr2=0x7f45740b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f45794d9640 1 -- 172.21.15.135:0/2856672779 shutdown_connections 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.646+0000 7f45794d9640 1 -- 172.21.15.135:0/2856672779 wait complete. 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.647+0000 7f45794d9640 1 Processor -- start 2024-04-17T06:10:05.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.647+0000 7f45794d9640 1 -- start start 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45794d9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409b470 0x7f45740b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45794d9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f457409bbe0 0x7f45740b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45794d9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f45740c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45794d9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f457406d910 con 0x7f457415ee30 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45794d9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f457406d790 con 0x7f457409bbe0 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45794d9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f45740ce640 con 0x7f457409b470 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f45740c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f45740c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55688/0 (socket says 172.21.15.135:55688) 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45737fe640 1 -- 172.21.15.135:0/2299665733 learned_addr learned my addr 172.21.15.135:0/2299665733 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:05.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f4572ffd640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409b470 0x7f45740b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.648+0000 7f45727fc640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f457409bbe0 0x7f45740b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45737fe640 1 -- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409b470 msgr2=0x7f45740b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45737fe640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409b470 0x7f45740b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45737fe640 1 -- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f457409bbe0 msgr2=0x7f45740b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45737fe640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f457409bbe0 0x7f45740b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45737fe640 1 -- 172.21.15.135:0/2299665733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45740b6c60 con 0x7f457415ee30 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45727fc640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f457409bbe0 0x7f45740b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45737fe640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f45740c8cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f456c00c450 tx=0x7f456c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f455b7fe640 1 -- 172.21.15.135:0/2299665733 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c018070 con 0x7f457415ee30 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f455b7fe640 1 -- 172.21.15.135:0/2299665733 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456c007500 con 0x7f457415ee30 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45740b63c0 con 0x7f457415ee30 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f455b7fe640 1 -- 172.21.15.135:0/2299665733 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c004a20 con 0x7f457415ee30 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.649+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45740b6980 con 0x7f457415ee30 2024-04-17T06:10:05.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.650+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4574074940 con 0x7f457415ee30 2024-04-17T06:10:05.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.650+0000 7f455b7fe640 1 -- 172.21.15.135:0/2299665733 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f456c01b070 con 0x7f457415ee30 2024-04-17T06:10:05.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.651+0000 7f455b7fe640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f456006c8f0 0x7f456006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.651+0000 7f455b7fe640 1 -- 172.21.15.135:0/2299665733 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f456c08b100 con 0x7f457415ee30 2024-04-17T06:10:05.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.651+0000 7f4572ffd640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f456006c8f0 0x7f456006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.651 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.651+0000 7f4572ffd640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f456006c8f0 0x7f456006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4564036000 tx=0x7f4564030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.655+0000 7f455b7fe640 1 -- 172.21.15.135:0/2299665733 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f456c058ae0 con 0x7f457415ee30 2024-04-17T06:10:05.828 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.830+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f457415df70 con 0x7f457415ee30 2024-04-17T06:10:05.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.831+0000 7f455b7fe640 1 -- 172.21.15.135:0/2299665733 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f456c05cde0 con 0x7f457415ee30 2024-04-17T06:10:05.829 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.832+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f456006c8f0 msgr2=0x7f456006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.832+0000 7f45794d9640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f456006c8f0 0x7f456006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4564036000 tx=0x7f4564030000 comp rx=0 tx=0).stop 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.832+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 msgr2=0x7f45740c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.832+0000 7f45794d9640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f45740c8cd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f456c00c450 tx=0x7f456c00c910 comp rx=0 tx=0).stop 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.833+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 shutdown_connections 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.833+0000 7f45794d9640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f456006c8f0 0x7f456006edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.833+0000 7f45794d9640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f457415ee30 0x7f45740c8cd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.833+0000 7f45794d9640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f457409bbe0 0x7f45740b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.833+0000 7f45794d9640 1 --2- 172.21.15.135:0/2299665733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f457409b470 0x7f45740b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.833+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 >> 172.21.15.135:0/2299665733 conn(0x7f45740b41a0 msgr2=0x7f457409a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:05.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.833+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 shutdown_connections 2024-04-17T06:10:05.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.833+0000 7f45794d9640 1 -- 172.21.15.135:0/2299665733 wait complete. 2024-04-17T06:10:05.842 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:10:05.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.950+0000 7fd2871bd640 1 Processor -- start 2024-04-17T06:10:05.948 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.950+0000 7fd2871bd640 1 -- start start 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd2871bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd28006b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd2871bd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2800cfe10 0x7fd28015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd2871bd640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2800b5ae0 con 0x7fd2800d0630 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd2871bd640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd2800b60a0 con 0x7fd2800cfe10 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd2871bd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd2800b5dc0 con 0x7fd28015ec70 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd284f59640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd28006b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd28575a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2800cfe10 0x7fd28015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd284f59640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd28006b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:50816/0 (socket says 172.21.15.135:50816) 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd284f59640 1 -- 172.21.15.135:0/24955834 learned_addr learned my addr 172.21.15.135:0/24955834 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:05.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd284f59640 1 -- 172.21.15.135:0/24955834 >> v2:172.21.15.190:3300/0 conn(0x7fd2800cfe10 msgr2=0x7fd28015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:05.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd284f59640 1 --2- 172.21.15.135:0/24955834 >> v2:172.21.15.190:3300/0 conn(0x7fd2800cfe10 0x7fd28015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd284f59640 1 -- 172.21.15.135:0/24955834 >> v1:172.21.15.190:6789/0 conn(0x7fd2800d0630 legacy=0x7fd2800cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd284f59640 1 -- 172.21.15.135:0/24955834 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2800b3e80 con 0x7fd28015ec70 2024-04-17T06:10:05.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.951+0000 7fd28575a640 1 --2- 172.21.15.135:0/24955834 >> v2:172.21.15.190:3300/0 conn(0x7fd2800cfe10 0x7fd28015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:10:05.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.952+0000 7fd284f59640 1 --2- 172.21.15.135:0/24955834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd28006b990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd274009870 tx=0x7fd27402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d73065bf7811d513 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.952+0000 7fd27f7fe640 1 -- 172.21.15.135:0/24955834 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274035020 con 0x7fd28015ec70 2024-04-17T06:10:05.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.952+0000 7fd27f7fe640 1 -- 172.21.15.135:0/24955834 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274031640 con 0x7fd28015ec70 2024-04-17T06:10:05.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.952+0000 7fd27f7fe640 1 -- 172.21.15.135:0/24955834 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27402d040 con 0x7fd28015ec70 2024-04-17T06:10:05.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.953+0000 7fd2871bd640 1 -- 172.21.15.135:0/24955834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 msgr2=0x7fd28006b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.953+0000 7fd2871bd640 1 --2- 172.21.15.135:0/24955834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd28006b990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd274009870 tx=0x7fd27402b8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.953+0000 7fd2871bd640 1 -- 172.21.15.135:0/24955834 shutdown_connections 2024-04-17T06:10:05.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.953+0000 7fd2871bd640 1 --2- 172.21.15.135:0/24955834 >> v2:172.21.15.190:3300/0 conn(0x7fd2800cfe10 0x7fd28015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.953+0000 7fd2871bd640 1 --2- 172.21.15.135:0/24955834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd28006b990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.953+0000 7fd2871bd640 1 -- 172.21.15.135:0/24955834 >> 172.21.15.135:0/24955834 conn(0x7fd2800cd7e0 msgr2=0x7fd2800cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:05.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.953+0000 7fd2871bd640 1 -- 172.21.15.135:0/24955834 shutdown_connections 2024-04-17T06:10:05.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.953+0000 7fd2871bd640 1 -- 172.21.15.135:0/24955834 wait complete. 2024-04-17T06:10:05.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.954+0000 7fd2871bd640 1 Processor -- start 2024-04-17T06:10:05.953 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.954+0000 7fd2871bd640 1 -- start start 2024-04-17T06:10:05.955 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd2871bd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2800cfe10 0x7fd280160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd2871bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2800d0630 0x7fd280152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd2871bd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd2801f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd2871bd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2800d02a0 con 0x7fd2800d0630 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd2871bd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd2800d0ab0 con 0x7fd2800cfe10 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd2871bd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd2800d0420 con 0x7fd28015ec70 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd27ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2800d0630 0x7fd280152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd27ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2800d0630 0x7fd280152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55702/0 (socket says 172.21.15.135:55702) 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd27ffff640 1 -- 172.21.15.135:0/884207624 learned_addr learned my addr 172.21.15.135:0/884207624 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd284f59640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2800cfe10 0x7fd280160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd28575a640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd2801f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd27ffff640 1 -- 172.21.15.135:0/884207624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 msgr2=0x7fd2801f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd27ffff640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd2801f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd27ffff640 1 -- 172.21.15.135:0/884207624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2800cfe10 msgr2=0x7fd280160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:05.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd27ffff640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2800cfe10 0x7fd280160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.955+0000 7fd27ffff640 1 -- 172.21.15.135:0/884207624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd280153310 con 0x7fd2800d0630 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.956+0000 7fd27ffff640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2800d0630 0x7fd280152dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00c400 tx=0x7fd26c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.956+0000 7fd27d7fa640 1 -- 172.21.15.135:0/884207624 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0052d0 con 0x7fd2800d0630 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.956+0000 7fd27d7fa640 1 -- 172.21.15.135:0/884207624 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26c005490 con 0x7fd2800d0630 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.956+0000 7fd27d7fa640 1 -- 172.21.15.135:0/884207624 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c007260 con 0x7fd2800d0630 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.956+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2801f5cc0 con 0x7fd2800d0630 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.956+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd280152010 con 0x7fd2800d0630 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.957+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd280074940 con 0x7fd2800d0630 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.957+0000 7fd27d7fa640 1 -- 172.21.15.135:0/884207624 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd26c007400 con 0x7fd2800d0630 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.958+0000 7fd27d7fa640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd26406c920 0x7fd26406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.958+0000 7fd284f59640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd26406c920 0x7fd26406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:05.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.958+0000 7fd27d7fa640 1 -- 172.21.15.135:0/884207624 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd26c08b460 con 0x7fd2800d0630 2024-04-17T06:10:05.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.958+0000 7fd284f59640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd26406c920 0x7fd26406ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd274036000 tx=0x7fd27403c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:05.960 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:05.962+0000 7fd27d7fa640 1 -- 172.21.15.135:0/884207624 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd26c059490 con 0x7fd2800d0630 2024-04-17T06:10:06.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.138+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd2800ce810 con 0x7fd2800d0630 2024-04-17T06:10:06.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.139+0000 7fd27d7fa640 1 -- 172.21.15.135:0/884207624 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd26c058d90 con 0x7fd2800d0630 2024-04-17T06:10:06.137 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:06.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd26406c920 msgr2=0x7fd26406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd26406c920 0x7fd26406ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd274036000 tx=0x7fd27403c040 comp rx=0 tx=0).stop 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2800d0630 msgr2=0x7fd280152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2800d0630 0x7fd280152dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00c400 tx=0x7fd26c00c8c0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 shutdown_connections 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd28015ec70 0x7fd2801f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd26406c920 0x7fd26406ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2800d0630 0x7fd280152dd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 --2- 172.21.15.135:0/884207624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2800cfe10 0x7fd280160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 >> 172.21.15.135:0/884207624 conn(0x7fd2800cd7e0 msgr2=0x7fd28006e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 shutdown_connections 2024-04-17T06:10:06.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.141+0000 7fd2871bd640 1 -- 172.21.15.135:0/884207624 wait complete. 2024-04-17T06:10:06.150 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T06:10:06.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.257+0000 7fc9e6dbd640 1 Processor -- start 2024-04-17T06:10:06.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.257+0000 7fc9e6dbd640 1 -- start start 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.258+0000 7fc9e6dbd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc9e009bbe0 0x7fc9e009b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.258+0000 7fc9e6dbd640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e006db80 con 0x7fc9e015ee30 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.258+0000 7fc9e6dbd640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e006de60 con 0x7fc9e00cd630 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.258+0000 7fc9e6dbd640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e00cce30 con 0x7fc9e009bbe0 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.258+0000 7fc9e4b59640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc9e015ee30 0x7fc9e009a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42298/0 (socket says 172.21.15.135:42298) 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.258+0000 7fc9e4b59640 1 -- 172.21.15.135:0/2549324622 learned_addr learned my addr 172.21.15.135:0/2549324622 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.258+0000 7fc9dffff640 1 --2- 172.21.15.135:0/2549324622 >> v2:172.21.15.135:3300/0 conn(0x7fc9e009bbe0 0x7fc9e009b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.258+0000 7fc9df7fe640 1 -- 172.21.15.135:0/2549324622 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804298533 0 0) 0x7fc9e006db80 con 0x7fc9e015ee30 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.259+0000 7fc9df7fe640 1 -- 172.21.15.135:0/2549324622 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c8003520 con 0x7fc9e015ee30 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.259+0000 7fc9dffff640 1 -- 172.21.15.135:0/2549324622 >> v1:172.21.15.190:6789/0 conn(0x7fc9e00cd630 legacy=0x7fc9e00cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.259+0000 7fc9dffff640 1 -- 172.21.15.135:0/2549324622 >> v1:172.21.15.135:6789/0 conn(0x7fc9e015ee30 legacy=0x7fc9e009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.259+0000 7fc9dffff640 1 -- 172.21.15.135:0/2549324622 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00cd110 con 0x7fc9e009bbe0 2024-04-17T06:10:06.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.259+0000 7fc9dffff640 1 --2- 172.21.15.135:0/2549324622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e009bbe0 0x7fc9e009b100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40097a0 tx=0x7fc9d402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8425693e57ad7903 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:06.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.259+0000 7fc9df7fe640 1 -- 172.21.15.135:0/2549324622 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4034070 con 0x7fc9e009bbe0 2024-04-17T06:10:06.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.259+0000 7fc9df7fe640 1 -- 172.21.15.135:0/2549324622 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4030610 con 0x7fc9e009bbe0 2024-04-17T06:10:06.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.260+0000 7fc9df7fe640 1 -- 172.21.15.135:0/2549324622 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40309e0 con 0x7fc9e009bbe0 2024-04-17T06:10:06.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.260+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2549324622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e009bbe0 msgr2=0x7fc9e009b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.259 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.260+0000 7fc9e6dbd640 1 --2- 172.21.15.135:0/2549324622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e009bbe0 0x7fc9e009b100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40097a0 tx=0x7fc9d402b960 comp rx=0 tx=0).stop 2024-04-17T06:10:06.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.260+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2549324622 shutdown_connections 2024-04-17T06:10:06.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.260+0000 7fc9e6dbd640 1 --2- 172.21.15.135:0/2549324622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e009bbe0 0x7fc9e009b100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.260+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2549324622 >> 172.21.15.135:0/2549324622 conn(0x7fc9e00b41a0 msgr2=0x7fc9e00b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:06.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.260+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2549324622 shutdown_connections 2024-04-17T06:10:06.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.260+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2549324622 wait complete. 2024-04-17T06:10:06.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.261+0000 7fc9e6dbd640 1 Processor -- start 2024-04-17T06:10:06.260 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.261+0000 7fc9e6dbd640 1 -- start start 2024-04-17T06:10:06.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e6dbd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e009bbe0 0x7fc9e014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e6dbd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e00cd630 0x7fc9e00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e6dbd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e015ee30 0x7fc9e00caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e6dbd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e006d910 con 0x7fc9e015ee30 2024-04-17T06:10:06.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e6dbd640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e006d790 con 0x7fc9e009bbe0 2024-04-17T06:10:06.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e6dbd640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e009b640 con 0x7fc9e00cd630 2024-04-17T06:10:06.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e535a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e015ee30 0x7fc9e00caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e535a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e015ee30 0x7fc9e00caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55708/0 (socket says 172.21.15.135:55708) 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e535a640 1 -- 172.21.15.135:0/2259070041 learned_addr learned my addr 172.21.15.135:0/2259070041 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e4b59640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e009bbe0 0x7fc9e014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9dffff640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e00cd630 0x7fc9e00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e535a640 1 -- 172.21.15.135:0/2259070041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e00cd630 msgr2=0x7fc9e00c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e535a640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e00cd630 0x7fc9e00c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e535a640 1 -- 172.21.15.135:0/2259070041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e009bbe0 msgr2=0x7fc9e014ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e535a640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e009bbe0 0x7fc9e014ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.262+0000 7fc9e535a640 1 -- 172.21.15.135:0/2259070041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e00c8c80 con 0x7fc9e015ee30 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.263+0000 7fc9e535a640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e015ee30 0x7fc9e00caaa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800c6b0 tx=0x7fc9d800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.263+0000 7fc9dd7fa640 1 -- 172.21.15.135:0/2259070041 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8003b70 con 0x7fc9e015ee30 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.263+0000 7fc9dd7fa640 1 -- 172.21.15.135:0/2259070041 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d8003d30 con 0x7fc9e015ee30 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.263+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e00c8f60 con 0x7fc9e015ee30 2024-04-17T06:10:06.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.263+0000 7fc9dd7fa640 1 -- 172.21.15.135:0/2259070041 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8005310 con 0x7fc9e015ee30 2024-04-17T06:10:06.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.263+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e014cf50 con 0x7fc9e015ee30 2024-04-17T06:10:06.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.264+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e0074940 con 0x7fc9e015ee30 2024-04-17T06:10:06.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.264+0000 7fc9dd7fa640 1 -- 172.21.15.135:0/2259070041 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9d80054b0 con 0x7fc9e015ee30 2024-04-17T06:10:06.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.265+0000 7fc9dd7fa640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc9cc06c920 0x7fc9cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.265+0000 7fc9dd7fa640 1 -- 172.21.15.135:0/2259070041 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc9d808bd00 con 0x7fc9e015ee30 2024-04-17T06:10:06.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.265+0000 7fc9e4b59640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc9cc06c920 0x7fc9cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.265+0000 7fc9e4b59640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc9cc06c920 0x7fc9cc06ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00011e0 tx=0x7fc9d0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:06.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.269+0000 7fc9dd7fa640 1 -- 172.21.15.135:0/2259070041 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9d8059dc0 con 0x7fc9e015ee30 2024-04-17T06:10:06.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.454+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc9e0070150 con 0x7fc9e015ee30 2024-04-17T06:10:06.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.455+0000 7fc9dd7fa640 1 -- 172.21.15.135:0/2259070041 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fc9d80596c0 con 0x7fc9e015ee30 2024-04-17T06:10:06.453 INFO:teuthology.orchestra.run.smithi135.stderr:listed 0 entries 2024-04-17T06:10:06.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc9cc06c920 msgr2=0x7fc9cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc9cc06c920 0x7fc9cc06ede0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00011e0 tx=0x7fc9d0009040 comp rx=0 tx=0).stop 2024-04-17T06:10:06.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e015ee30 msgr2=0x7fc9e00caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e015ee30 0x7fc9e00caaa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800c6b0 tx=0x7fc9d800cb70 comp rx=0 tx=0).stop 2024-04-17T06:10:06.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 shutdown_connections 2024-04-17T06:10:06.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc9cc06c920 0x7fc9cc06ede0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9e015ee30 0x7fc9e00caaa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e00cd630 0x7fc9e00c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 --2- 172.21.15.135:0/2259070041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e009bbe0 0x7fc9e014ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 >> 172.21.15.135:0/2259070041 conn(0x7fc9e00b41a0 msgr2=0x7fc9e00d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:06.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 shutdown_connections 2024-04-17T06:10:06.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.457+0000 7fc9e6dbd640 1 -- 172.21.15.135:0/2259070041 wait complete. 2024-04-17T06:10:06.466 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-17T06:10:06.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:06.466 DEBUG:teuthology.orchestra.run.smithi135:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-17T06:10:06.496 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:10:06.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:06.496 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T06:10:06.552 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:06.553 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:06.572 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:06.572 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:06.632 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:06.632 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:06.690 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:06.794 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.796+0000 7f7f50a1b640 1 Processor -- start 2024-04-17T06:10:06.795 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.796+0000 7f7f50a1b640 1 -- start start 2024-04-17T06:10:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.797+0000 7f7f50a1b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7f4c0d43d0 0x7f7f4c0d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.797+0000 7f7f50a1b640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c0d30a0 con 0x7f7f4c15ec00 2024-04-17T06:10:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.797+0000 7f7f50a1b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c0d3330 con 0x7f7f4c0d1810 2024-04-17T06:10:06.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.797+0000 7f7f50a1b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4c15efe0 con 0x7f7f4c0d43d0 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4a59c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7f4c15ec00 0x7f7f4c0d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42306/0 (socket says 172.21.15.135:42306) 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4a59c640 1 -- 172.21.15.135:0/4178567912 learned_addr learned my addr 172.21.15.135:0/4178567912 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3883636994 0 0) 0x7f7f4c0d30a0 con 0x7f7f4c15ec00 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f38003520 con 0x7f7f4c15ec00 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f49d9b640 1 --2- 172.21.15.135:0/4178567912 >> v2:172.21.15.190:3300/0 conn(0x7f7f4c0d43d0 0x7f7f4c0d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2595784845 0 0) 0x7f7f38003520 con 0x7f7f4c15ec00 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 >> v2:172.21.15.190:3300/0 conn(0x7f7f4c0d43d0 msgr2=0x7f7f4c0d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 --2- 172.21.15.135:0/4178567912 >> v2:172.21.15.190:3300/0 conn(0x7f7f4c0d43d0 0x7f7f4c0d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 >> v1:172.21.15.190:6789/0 conn(0x7f7f4c0d1810 legacy=0x7f7f4c0b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c0d3610 con 0x7f7f4c15ec00 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f7f440019f0 con 0x7f7f4c15ec00 2024-04-17T06:10:06.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 >> v1:172.21.15.135:6789/0 conn(0x7f7f4c15ec00 legacy=0x7f7f4c0d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f38004800 0x7f7f38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f380051c0 0x7f7f3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c15efe0 con 0x7f7f380039b0 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3800c070 con 0x7f7f380051c0 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3800c270 con 0x7f7f38004800 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4a59c640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f49d9b640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f38004800 0x7f7f38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4ad9d640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f380051c0 0x7f7f3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4a59c640 1 -- 172.21.15.135:0/4178567912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f38004800 msgr2=0x7f7f38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4a59c640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f38004800 0x7f7f38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4a59c640 1 -- 172.21.15.135:0/4178567912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f380051c0 msgr2=0x7f7f3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4a59c640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f380051c0 0x7f7f3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4ad9d640 1 -- 172.21.15.135:0/4178567912 reap_dead start 2024-04-17T06:10:06.800 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.798+0000 7f7f4a59c640 1 -- 172.21.15.135:0/4178567912 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f38003520 con 0x7f7f380039b0 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.799+0000 7f7f4a59c640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f38003dd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400a850 tx=0x7f7f38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4777b360e965ccf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.799+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400a4a0 con 0x7f7f380039b0 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.799+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f44034650 con 0x7f7f380039b0 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.799+0000 7f7f4959a640 1 -- 172.21.15.135:0/4178567912 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44034a80 con 0x7f7f380039b0 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.800+0000 7f7f50a1b640 1 -- 172.21.15.135:0/4178567912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 msgr2=0x7f7f38003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.800+0000 7f7f50a1b640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f38003dd0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400a850 tx=0x7f7f38005830 comp rx=0 tx=0).stop 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.800+0000 7f7f50a1b640 1 -- 172.21.15.135:0/4178567912 shutdown_connections 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.800+0000 7f7f50a1b640 1 --2- 172.21.15.135:0/4178567912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f38003dd0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.800+0000 7f7f50a1b640 1 -- 172.21.15.135:0/4178567912 >> 172.21.15.135:0/4178567912 conn(0x7f7f4c06cc90 msgr2=0x7f7f4c06d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.800+0000 7f7f50a1b640 1 -- 172.21.15.135:0/4178567912 shutdown_connections 2024-04-17T06:10:06.801 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.800+0000 7f7f50a1b640 1 -- 172.21.15.135:0/4178567912 wait complete. 2024-04-17T06:10:06.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.801+0000 7f7f50a1b640 1 Processor -- start 2024-04-17T06:10:06.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.801+0000 7f7f50a1b640 1 -- start start 2024-04-17T06:10:06.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f50a1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f4c1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f50a1b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f4c0ca950 0x7f7f4c0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f50a1b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f4c160280 0x7f7f4c160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.804 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f50a1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c0d1c60 con 0x7f7f380039b0 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f50a1b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c0d3f40 con 0x7f7f4c160280 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f50a1b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4c0d3bd0 con 0x7f7f4c0ca950 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f4a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f4c1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f4a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f4c1f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55730/0 (socket says 172.21.15.135:55730) 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f4a59c640 1 -- 172.21.15.135:0/940954249 learned_addr learned my addr 172.21.15.135:0/940954249 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f49d9b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f4c0ca950 0x7f7f4c0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.802+0000 7f7f4ad9d640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f4c160280 0x7f7f4c160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f4a59c640 1 -- 172.21.15.135:0/940954249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f4c0ca950 msgr2=0x7f7f4c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f4a59c640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f4c0ca950 0x7f7f4c0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f4a59c640 1 -- 172.21.15.135:0/940954249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f4c160280 msgr2=0x7f7f4c160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f4a59c640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f4c160280 0x7f7f4c160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f4a59c640 1 -- 172.21.15.135:0/940954249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c160fb0 con 0x7f7f380039b0 2024-04-17T06:10:06.805 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f4a59c640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f4c1f5220 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400a850 tx=0x7f7f44001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f42ffd640 1 -- 172.21.15.135:0/940954249 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44039040 con 0x7f7f380039b0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f42ffd640 1 -- 172.21.15.135:0/940954249 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f44004b60 con 0x7f7f380039b0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f42ffd640 1 -- 172.21.15.135:0/940954249 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44034800 con 0x7f7f380039b0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c161290 con 0x7f7f380039b0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.803+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1f5760 con 0x7f7f380039b0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.804+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4c0b74f0 con 0x7f7f380039b0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.804+0000 7f7f42ffd640 1 -- 172.21.15.135:0/940954249 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f440349a0 con 0x7f7f380039b0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.805+0000 7f7f42ffd640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7f2c06c920 0x7f7f2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.805+0000 7f7f49d9b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7f2c06c920 0x7f7f2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.805+0000 7f7f42ffd640 1 -- 172.21.15.135:0/940954249 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7f440ade20 con 0x7f7f380039b0 2024-04-17T06:10:06.806 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.805+0000 7f7f49d9b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7f2c06c920 0x7f7f2c06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c1556c0 tx=0x7f7f34008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:06.807 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.809+0000 7f7f42ffd640 1 -- 172.21.15.135:0/940954249 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f4c0b74f0 con 0x7f7f380039b0 2024-04-17T06:10:06.983 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.985+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f4c074940 con 0x7f7f380039b0 2024-04-17T06:10:06.984 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.986+0000 7f7f42ffd640 1 -- 172.21.15.135:0/940954249 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18950 (secure 0 0 0) 0x7f7f4407bee0 con 0x7f7f380039b0 2024-04-17T06:10:06.984 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:06.984 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:01.008924+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:06.986 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7f2c06c920 msgr2=0x7f7f2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7f2c06c920 0x7f7f2c06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c1556c0 tx=0x7f7f34008040 comp rx=0 tx=0).stop 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 msgr2=0x7f7f4c1f5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f4c1f5220 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400a850 tx=0x7f7f44001dc0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 shutdown_connections 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f4c160280 0x7f7f4c160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f4c0ca950 0x7f7f4c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7f2c06c920 0x7f7f2c06ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 --2- 172.21.15.135:0/940954249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f380039b0 0x7f7f4c1f5220 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 >> 172.21.15.135:0/940954249 conn(0x7f7f4c06cc90 msgr2=0x7f7f4c0b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 shutdown_connections 2024-04-17T06:10:06.987 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:06.988+0000 7f7f50a1b640 1 -- 172.21.15.135:0/940954249 wait complete. 2024-04-17T06:10:06.998 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:10:06.998 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:10:07.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.107+0000 7fb4bee55640 1 Processor -- start 2024-04-17T06:10:07.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.107+0000 7fb4bee55640 1 -- start start 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bee55640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b8158e30 0x7fb4b815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bee55640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb4b806f610 0x7fb4b815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bee55640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b815fb80 con 0x7fb4b8158a50 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bee55640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b815b840 con 0x7fb4b806f610 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bee55640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b806fac0 con 0x7fb4b8158e30 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bd3f2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb4b806f610 0x7fb4b815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bcbf1640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb4b8158a50 0x7fb4b80b3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42310/0 (socket says 172.21.15.135:42310) 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bd3f2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb4b806f610 0x7fb4b815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55744/0 (socket says 172.21.15.135:55744) 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bcbf1640 1 -- 172.21.15.135:0/1914563042 learned_addr learned my addr 172.21.15.135:0/1914563042 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:07.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4b7fff640 1 --2- 172.21.15.135:0/1914563042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b8158e30 0x7fb4b815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bd3f2640 1 -- 172.21.15.135:0/1914563042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b8158e30 msgr2=0x7fb4b815ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bd3f2640 1 --2- 172.21.15.135:0/1914563042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b8158e30 0x7fb4b815ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bd3f2640 1 -- 172.21.15.135:0/1914563042 >> v1:172.21.15.135:6789/0 conn(0x7fb4b8158a50 legacy=0x7fb4b80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4bd3f2640 1 -- 172.21.15.135:0/1914563042 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b815f2e0 con 0x7fb4b806f610 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.108+0000 7fb4b77fe640 1 -- 172.21.15.135:0/1914563042 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2322225938 0 0) 0x7fb4b815fb80 con 0x7fb4b8158a50 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.109+0000 7fb4bd3f2640 1 --2- 172.21.15.135:0/1914563042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b806f610 0x7fb4b815bcd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000aea0 tx=0x7fb4b002cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222d2bcefe7a3d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.109+0000 7fb4b77fe640 1 -- 172.21.15.135:0/1914563042 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b002db40 con 0x7fb4b806f610 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.109+0000 7fb4b77fe640 1 -- 172.21.15.135:0/1914563042 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b002dce0 con 0x7fb4b806f610 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.109+0000 7fb4b77fe640 1 -- 172.21.15.135:0/1914563042 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0008660 con 0x7fb4b806f610 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.110+0000 7fb4bee55640 1 -- 172.21.15.135:0/1914563042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b806f610 msgr2=0x7fb4b815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.110+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1914563042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b806f610 0x7fb4b815bcd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000aea0 tx=0x7fb4b002cfa0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.110+0000 7fb4bee55640 1 -- 172.21.15.135:0/1914563042 shutdown_connections 2024-04-17T06:10:07.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.110+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1914563042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b8158e30 0x7fb4b815ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.110+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1914563042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b806f610 0x7fb4b815bcd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.110+0000 7fb4bee55640 1 -- 172.21.15.135:0/1914563042 >> 172.21.15.135:0/1914563042 conn(0x7fb4b80d0520 msgr2=0x7fb4b80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:07.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.110+0000 7fb4bee55640 1 -- 172.21.15.135:0/1914563042 shutdown_connections 2024-04-17T06:10:07.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.110+0000 7fb4bee55640 1 -- 172.21.15.135:0/1914563042 wait complete. 2024-04-17T06:10:07.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.111+0000 7fb4bee55640 1 Processor -- start 2024-04-17T06:10:07.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.111+0000 7fb4bee55640 1 -- start start 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bee55640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b806f610 0x7fb4b81f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bee55640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4b8158a50 0x7fb4b80b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bee55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b8158e30 0x7fb4b80b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bee55640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b80b3ca0 con 0x7fb4b8158e30 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bee55640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b815f8a0 con 0x7fb4b8158a50 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bee55640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b8149be0 con 0x7fb4b806f610 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b8158e30 0x7fb4b80b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b8158e30 0x7fb4b80b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55752/0 (socket says 172.21.15.135:55752) 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 -- 172.21.15.135:0/1935858565 learned_addr learned my addr 172.21.15.135:0/1935858565 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bcbf1640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b806f610 0x7fb4b81f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4b7fff640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4b8158a50 0x7fb4b80b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 -- 172.21.15.135:0/1935858565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b806f610 msgr2=0x7fb4b81f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b806f610 0x7fb4b81f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 -- 172.21.15.135:0/1935858565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4b8158a50 msgr2=0x7fb4b80b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4b8158a50 0x7fb4b80b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 -- 172.21.15.135:0/1935858565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b8149ec0 con 0x7fb4b8158e30 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4bd3f2640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b8158e30 0x7fb4b80b1150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0037040 tx=0x7fb4b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.112+0000 7fb4b57fa640 1 -- 172.21.15.135:0/1935858565 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0044030 con 0x7fb4b8158e30 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.113+0000 7fb4b57fa640 1 -- 172.21.15.135:0/1935858565 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b0040470 con 0x7fb4b8158e30 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.113+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b814a1a0 con 0x7fb4b8158e30 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.113+0000 7fb4b57fa640 1 -- 172.21.15.135:0/1935858565 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0040860 con 0x7fb4b8158e30 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.113+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b8149880 con 0x7fb4b8158e30 2024-04-17T06:10:07.115 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.113+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b8074940 con 0x7fb4b8158e30 2024-04-17T06:10:07.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.114+0000 7fb4b57fa640 1 -- 172.21.15.135:0/1935858565 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4b0047070 con 0x7fb4b8158e30 2024-04-17T06:10:07.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.114+0000 7fb4b57fa640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb49c06c8f0 0x7fb49c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.115+0000 7fb4b57fa640 1 -- 172.21.15.135:0/1935858565 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fb4b00ae880 con 0x7fb4b8158e30 2024-04-17T06:10:07.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.115+0000 7fb4bcbf1640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb49c06c8f0 0x7fb49c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.115+0000 7fb4bcbf1640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb49c06c8f0 0x7fb49c06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000f60 tx=0x7fb4ac007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.119+0000 7fb4b57fa640 1 -- 172.21.15.135:0/1935858565 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4b004e070 con 0x7fb4b8158e30 2024-04-17T06:10:07.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.294+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4b8072060 con 0x7fb4b8158e30 2024-04-17T06:10:07.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.294+0000 7fb4b57fa640 1 -- 172.21.15.135:0/1935858565 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fb4b007c380 con 0x7fb4b8158e30 2024-04-17T06:10:07.293 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb49c06c8f0 msgr2=0x7fb49c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb49c06c8f0 0x7fb49c06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000f60 tx=0x7fb4ac007d50 comp rx=0 tx=0).stop 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b8158e30 msgr2=0x7fb4b80b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b8158e30 0x7fb4b80b1150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0037040 tx=0x7fb4b0004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 shutdown_connections 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb49c06c8f0 0x7fb49c06edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4b8158e30 0x7fb4b80b1150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb4b8158a50 0x7fb4b80b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.296+0000 7fb4bee55640 1 --2- 172.21.15.135:0/1935858565 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb4b806f610 0x7fb4b81f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.297+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 >> 172.21.15.135:0/1935858565 conn(0x7fb4b80d0520 msgr2=0x7fb4b80b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:07.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.297+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 shutdown_connections 2024-04-17T06:10:07.296 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.297+0000 7fb4bee55640 1 -- 172.21.15.135:0/1935858565 wait complete. 2024-04-17T06:10:07.306 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:10:07.411 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.413+0000 7fdc13be8640 1 Processor -- start 2024-04-17T06:10:07.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.414+0000 7fdc13be8640 1 -- start start 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.414+0000 7fdc13be8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdc0c15ec00 0x7fdc0c0d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.414+0000 7fdc13be8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdc0c0d43d0 0x7fdc0c0d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.414+0000 7fdc13be8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc0c0d3330 con 0x7fdc0c0d1810 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.414+0000 7fdc13be8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fdc0c15efe0 con 0x7fdc0c0d43d0 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.414+0000 7fdc13be8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fdc0c0d3f40 con 0x7fdc0c15ec00 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.415+0000 7fdc11183640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fdc0c0d43d0 0x7fdc0c0d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.415+0000 7fdc12185640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fdc0c0d1810 0x7fdc0c0730c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42320/0 (socket says 172.21.15.135:42320) 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.415+0000 7fdc12185640 1 -- 172.21.15.135:0/2251073021 learned_addr learned my addr 172.21.15.135:0/2251073021 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:07.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.415+0000 7fdc11984640 1 --2- 172.21.15.135:0/2251073021 >> v2:172.21.15.190:3300/0 conn(0x7fdc0c15ec00 0x7fdc0c0d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.415+0000 7fdc11183640 1 -- 172.21.15.135:0/2251073021 >> v2:172.21.15.190:3300/0 conn(0x7fdc0c15ec00 msgr2=0x7fdc0c0d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:07.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.415+0000 7fdc11183640 1 --2- 172.21.15.135:0/2251073021 >> v2:172.21.15.190:3300/0 conn(0x7fdc0c15ec00 0x7fdc0c0d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.415+0000 7fdc11183640 1 -- 172.21.15.135:0/2251073021 >> v1:172.21.15.135:6789/0 conn(0x7fdc0c0d1810 legacy=0x7fdc0c0730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.415+0000 7fdc11183640 1 -- 172.21.15.135:0/2251073021 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c0d3610 con 0x7fdc0c0d43d0 2024-04-17T06:10:07.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc11183640 1 --2- 172.21.15.135:0/2251073021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d43d0 0x7fdc0c0d12d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c06db40 tx=0x7fdc0802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c460d5a85158a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc10982640 1 -- 172.21.15.135:0/2251073021 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08034070 con 0x7fdc0c0d43d0 2024-04-17T06:10:07.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc10982640 1 -- 172.21.15.135:0/2251073021 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc08030610 con 0x7fdc0c0d43d0 2024-04-17T06:10:07.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc10982640 1 -- 172.21.15.135:0/2251073021 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc080309e0 con 0x7fdc0c0d43d0 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc13be8640 1 -- 172.21.15.135:0/2251073021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d43d0 msgr2=0x7fdc0c0d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc13be8640 1 --2- 172.21.15.135:0/2251073021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d43d0 0x7fdc0c0d12d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c06db40 tx=0x7fdc0802b960 comp rx=0 tx=0).stop 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc13be8640 1 -- 172.21.15.135:0/2251073021 shutdown_connections 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc13be8640 1 --2- 172.21.15.135:0/2251073021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d43d0 0x7fdc0c0d12d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc13be8640 1 --2- 172.21.15.135:0/2251073021 >> v2:172.21.15.190:3300/0 conn(0x7fdc0c15ec00 0x7fdc0c0d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc13be8640 1 -- 172.21.15.135:0/2251073021 >> 172.21.15.135:0/2251073021 conn(0x7fdc0c06cc90 msgr2=0x7fdc0c06d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc13be8640 1 -- 172.21.15.135:0/2251073021 shutdown_connections 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.416+0000 7fdc13be8640 1 -- 172.21.15.135:0/2251073021 wait complete. 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc13be8640 1 Processor -- start 2024-04-17T06:10:07.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc13be8640 1 -- start start 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc13be8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d1810 0x7fdc0c151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc13be8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0d43d0 0x7fdc0c1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc13be8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15ec00 0x7fdc0c0d4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc13be8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c0d3bd0 con 0x7fdc0c0d1810 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc13be8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c0d1c60 con 0x7fdc0c15ec00 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc13be8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0c0d2ed0 con 0x7fdc0c0d43d0 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc11984640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d1810 0x7fdc0c151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc11984640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d1810 0x7fdc0c151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55774/0 (socket says 172.21.15.135:55774) 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.418+0000 7fdc11984640 1 -- 172.21.15.135:0/3878443850 learned_addr learned my addr 172.21.15.135:0/3878443850 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:07.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc11183640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0d43d0 0x7fdc0c1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc12185640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15ec00 0x7fdc0c0d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc11984640 1 -- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0d43d0 msgr2=0x7fdc0c1603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc11984640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0d43d0 0x7fdc0c1603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc11984640 1 -- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15ec00 msgr2=0x7fdc0c0d4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc11984640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15ec00 0x7fdc0c0d4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc11984640 1 -- 172.21.15.135:0/3878443850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c0ca8e0 con 0x7fdc0c0d1810 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc11183640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0d43d0 0x7fdc0c1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc11984640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d1810 0x7fdc0c151230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000e8a0 tx=0x7fdc0000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdbfe7fc640 1 -- 172.21.15.135:0/3878443850 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00017070 con 0x7fdc0c0d1810 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdbfe7fc640 1 -- 172.21.15.135:0/3878443850 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc0000c940 con 0x7fdc0c0d1810 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c0cabc0 con 0x7fdc0c0d1810 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdbfe7fc640 1 -- 172.21.15.135:0/3878443850 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000cd30 con 0x7fdc0c0d1810 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.419+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c0cb180 con 0x7fdc0c0d1810 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.420+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0c074940 con 0x7fdc0c0d1810 2024-04-17T06:10:07.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.420+0000 7fdbfe7fc640 1 -- 172.21.15.135:0/3878443850 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc0001a070 con 0x7fdc0c0d1810 2024-04-17T06:10:07.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.421+0000 7fdbfe7fc640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdbf006c8f0 0x7fdbf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.421+0000 7fdbfe7fc640 1 -- 172.21.15.135:0/3878443850 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fdc0008c190 con 0x7fdc0c0d1810 2024-04-17T06:10:07.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.421+0000 7fdc11183640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdbf006c8f0 0x7fdbf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.421+0000 7fdc11183640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdbf006c8f0 0x7fdbf006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc08036000 tx=0x7fdc0803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.425+0000 7fdbfe7fc640 1 -- 172.21.15.135:0/3878443850 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc0005a1c0 con 0x7fdc0c0d1810 2024-04-17T06:10:07.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.598+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc0c06bd20 con 0x7fdc0c0d1810 2024-04-17T06:10:07.597 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.599+0000 7fdbfe7fc640 1 -- 172.21.15.135:0/3878443850 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fdc00059ac0 con 0x7fdc0c0d1810 2024-04-17T06:10:07.597 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:07.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdbf006c8f0 msgr2=0x7fdbf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdbf006c8f0 0x7fdbf006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdc08036000 tx=0x7fdc0803b040 comp rx=0 tx=0).stop 2024-04-17T06:10:07.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d1810 msgr2=0x7fdc0c151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d1810 0x7fdc0c151230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000e8a0 tx=0x7fdc0000ed60 comp rx=0 tx=0).stop 2024-04-17T06:10:07.599 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 shutdown_connections 2024-04-17T06:10:07.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdc0c15ec00 0x7fdc0c0d4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdc0c0d43d0 0x7fdc0c1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fdbf006c8f0 0x7fdbf006edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 --2- 172.21.15.135:0/3878443850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc0c0d1810 0x7fdc0c151230 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 >> 172.21.15.135:0/3878443850 conn(0x7fdc0c06cc90 msgr2=0x7fdc0c0b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:07.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 shutdown_connections 2024-04-17T06:10:07.600 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.601+0000 7fdc13be8640 1 -- 172.21.15.135:0/3878443850 wait complete. 2024-04-17T06:10:07.610 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:10:07.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.720+0000 7f4534293640 1 Processor -- start 2024-04-17T06:10:07.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.720+0000 7f4534293640 1 -- start start 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f4534293640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f452c15cc60 0x7f452c0d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f4534293640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f452c0d0490 0x7f452c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f4534293640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f452c15d040 con 0x7f452c09af30 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f4534293640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f452c09b380 con 0x7f452c0d0490 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f4534293640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f452c0d13c0 con 0x7f452c15cc60 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f453182e640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f452c0d0490 0x7f452c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f4532830640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f452c09af30 0x7f452c0cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42324/0 (socket says 172.21.15.135:42324) 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f4532830640 1 -- 172.21.15.135:0/1839876288 learned_addr learned my addr 172.21.15.135:0/1839876288 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f453182e640 1 -- 172.21.15.135:0/1839876288 >> v2:172.21.15.190:3300/0 conn(0x7f452c15cc60 msgr2=0x7f452c0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:07.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f453182e640 1 --2- 172.21.15.135:0/1839876288 >> v2:172.21.15.190:3300/0 conn(0x7f452c15cc60 0x7f452c0d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f453182e640 1 -- 172.21.15.135:0/1839876288 >> v1:172.21.15.135:6789/0 conn(0x7f452c09af30 legacy=0x7f452c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f453102d640 1 -- 172.21.15.135:0/1839876288 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1564474038 0 0) 0x7f452c15d040 con 0x7f452c09af30 2024-04-17T06:10:07.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.721+0000 7f453182e640 1 -- 172.21.15.135:0/1839876288 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c0b1d20 con 0x7f452c0d0490 2024-04-17T06:10:07.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.722+0000 7f453182e640 1 --2- 172.21.15.135:0/1839876288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c0d0490 0x7f452c09a9f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f452c0730e0 tx=0x7f451c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db77ec07ce75c3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.722+0000 7f453102d640 1 -- 172.21.15.135:0/1839876288 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c035020 con 0x7f452c0d0490 2024-04-17T06:10:07.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.722+0000 7f453102d640 1 -- 172.21.15.135:0/1839876288 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f451c031830 con 0x7f452c0d0490 2024-04-17T06:10:07.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.722+0000 7f453102d640 1 -- 172.21.15.135:0/1839876288 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c031c00 con 0x7f452c0d0490 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.722+0000 7f4534293640 1 -- 172.21.15.135:0/1839876288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c0d0490 msgr2=0x7f452c09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.722+0000 7f4534293640 1 --2- 172.21.15.135:0/1839876288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c0d0490 0x7f452c09a9f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f452c0730e0 tx=0x7f451c02b8d0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.723+0000 7f4534293640 1 -- 172.21.15.135:0/1839876288 shutdown_connections 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.723+0000 7f4534293640 1 --2- 172.21.15.135:0/1839876288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c0d0490 0x7f452c09a9f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.723+0000 7f4534293640 1 --2- 172.21.15.135:0/1839876288 >> v2:172.21.15.190:3300/0 conn(0x7f452c15cc60 0x7f452c0d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.723+0000 7f4534293640 1 -- 172.21.15.135:0/1839876288 >> 172.21.15.135:0/1839876288 conn(0x7f452c0b41a0 msgr2=0x7f452c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.723+0000 7f4534293640 1 -- 172.21.15.135:0/1839876288 shutdown_connections 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.723+0000 7f4534293640 1 -- 172.21.15.135:0/1839876288 wait complete. 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.724+0000 7f4534293640 1 Processor -- start 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.724+0000 7f4534293640 1 -- start start 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.724+0000 7f4534293640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f452c09af30 0x7f452c15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.724+0000 7f4534293640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f452c0d0490 0x7f452c0cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4534293640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c15cc60 0x7f452c158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4534293640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f452c0d10d0 con 0x7f452c15cc60 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4534293640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f452c0b22e0 con 0x7f452c0d0490 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4534293640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f452c0d16a0 con 0x7f452c09af30 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c15cc60 0x7f452c158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c15cc60 0x7f452c158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55794/0 (socket says 172.21.15.135:55794) 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 -- 172.21.15.135:0/3024829446 learned_addr learned my addr 172.21.15.135:0/3024829446 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f453182e640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f452c0d0490 0x7f452c0cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f453202f640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f452c09af30 0x7f452c15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 -- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f452c09af30 msgr2=0x7f452c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f452c09af30 0x7f452c15f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 -- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f452c0d0490 msgr2=0x7f452c0cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f452c0d0490 0x7f452c0cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 -- 172.21.15.135:0/3024829446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c15e740 con 0x7f452c15cc60 2024-04-17T06:10:07.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f453182e640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f452c0d0490 0x7f452c0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:07.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4532830640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c15cc60 0x7f452c158c00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f452400c440 tx=0x7f452400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4522ffd640 1 -- 172.21.15.135:0/3024829446 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45240074e0 con 0x7f452c15cc60 2024-04-17T06:10:07.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4522ffd640 1 -- 172.21.15.135:0/3024829446 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45240047e0 con 0x7f452c15cc60 2024-04-17T06:10:07.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c15ea20 con 0x7f452c15cc60 2024-04-17T06:10:07.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4522ffd640 1 -- 172.21.15.135:0/3024829446 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524004bd0 con 0x7f452c15cc60 2024-04-17T06:10:07.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.725+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c15efe0 con 0x7f452c15cc60 2024-04-17T06:10:07.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.726+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c074940 con 0x7f452c15cc60 2024-04-17T06:10:07.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.727+0000 7f4522ffd640 1 -- 172.21.15.135:0/3024829446 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4524011070 con 0x7f452c15cc60 2024-04-17T06:10:07.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.727+0000 7f4522ffd640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f451006c920 0x7f451006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:07.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.727+0000 7f4522ffd640 1 -- 172.21.15.135:0/3024829446 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f452408b990 con 0x7f452c15cc60 2024-04-17T06:10:07.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.727+0000 7f453202f640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f451006c920 0x7f451006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:07.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.728+0000 7f453202f640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f451006c920 0x7f451006ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f452800a400 tx=0x7f452800d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:07.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.731+0000 7f4522ffd640 1 -- 172.21.15.135:0/3024829446 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4524059a50 con 0x7f452c15cc60 2024-04-17T06:10:07.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.906+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f452c071930 con 0x7f452c15cc60 2024-04-17T06:10:07.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.906+0000 7f4522ffd640 1 -- 172.21.15.135:0/3024829446 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f4524059350 con 0x7f452c15cc60 2024-04-17T06:10:07.904 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:10:07.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f451006c920 msgr2=0x7f451006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f451006c920 0x7f451006ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f452800a400 tx=0x7f452800d040 comp rx=0 tx=0).stop 2024-04-17T06:10:07.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c15cc60 msgr2=0x7f452c158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c15cc60 0x7f452c158c00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f452400c440 tx=0x7f452400c900 comp rx=0 tx=0).stop 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 shutdown_connections 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f451006c920 0x7f451006ede0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f452c15cc60 0x7f452c158c00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f452c0d0490 0x7f452c0cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 --2- 172.21.15.135:0/3024829446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f452c09af30 0x7f452c15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 >> 172.21.15.135:0/3024829446 conn(0x7f452c0b41a0 msgr2=0x7f452c074e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 shutdown_connections 2024-04-17T06:10:07.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:07.908+0000 7f4534293640 1 -- 172.21.15.135:0/3024829446 wait complete. 2024-04-17T06:10:07.917 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:10:08.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.026+0000 7f3b0fa81640 1 Processor -- start 2024-04-17T06:10:08.024 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.026+0000 7f3b0fa81640 1 -- start start 2024-04-17T06:10:08.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0fa81640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3b08162190 0x7f3b0806d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0fa81640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b080d1cf0 0x7f3b0809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0fa81640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b080d2180 con 0x7f3b0809af30 2024-04-17T06:10:08.025 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0fa81640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0809b360 con 0x7f3b080d1cf0 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0fa81640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3b080ccd00 con 0x7f3b08162190 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0d01c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b080d1cf0 0x7f3b0809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0d01c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f3b080d1cf0 0x7f3b0809a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55802/0 (socket says 172.21.15.135:55802) 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0d01c640 1 -- 172.21.15.135:0/3215699075 learned_addr learned my addr 172.21.15.135:0/3215699075 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0d81d640 1 --2- 172.21.15.135:0/3215699075 >> v2:172.21.15.190:3300/0 conn(0x7f3b08162190 0x7f3b0806d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0d01c640 1 -- 172.21.15.135:0/3215699075 >> v2:172.21.15.190:3300/0 conn(0x7f3b08162190 msgr2=0x7f3b0806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0d01c640 1 --2- 172.21.15.135:0/3215699075 >> v2:172.21.15.190:3300/0 conn(0x7f3b08162190 0x7f3b0806d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0d01c640 1 -- 172.21.15.135:0/3215699075 >> v1:172.21.15.190:6789/0 conn(0x7f3b0809af30 legacy=0x7f3b080d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:08.026 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.027+0000 7f3b0d01c640 1 -- 172.21.15.135:0/3215699075 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0806ddc0 con 0x7f3b080d1cf0 2024-04-17T06:10:08.027 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.028+0000 7f3b0d01c640 1 --2- 172.21.15.135:0/3215699075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080d1cf0 0x7f3b0809a9f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3b080739d0 tx=0x7f3af802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9231293d6a89a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.028+0000 7f3b0c81b640 1 -- 172.21.15.135:0/3215699075 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8035020 con 0x7f3b080d1cf0 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.028+0000 7f3b0c81b640 1 -- 172.21.15.135:0/3215699075 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af8031870 con 0x7f3b080d1cf0 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.028+0000 7f3b0c81b640 1 -- 172.21.15.135:0/3215699075 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8031c40 con 0x7f3b080d1cf0 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.029+0000 7f3b0fa81640 1 -- 172.21.15.135:0/3215699075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080d1cf0 msgr2=0x7f3b0809a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.029+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/3215699075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080d1cf0 0x7f3b0809a9f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3b080739d0 tx=0x7f3af802b910 comp rx=0 tx=0).stop 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.029+0000 7f3b0fa81640 1 -- 172.21.15.135:0/3215699075 shutdown_connections 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.029+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/3215699075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b080d1cf0 0x7f3b0809a9f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.029+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/3215699075 >> v2:172.21.15.190:3300/0 conn(0x7f3b08162190 0x7f3b0806d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.028 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.029+0000 7f3b0fa81640 1 -- 172.21.15.135:0/3215699075 >> 172.21.15.135:0/3215699075 conn(0x7f3b080b41a0 msgr2=0x7f3b080b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:08.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.029+0000 7f3b0fa81640 1 -- 172.21.15.135:0/3215699075 shutdown_connections 2024-04-17T06:10:08.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.029+0000 7f3b0fa81640 1 -- 172.21.15.135:0/3215699075 wait complete. 2024-04-17T06:10:08.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.030+0000 7f3b0fa81640 1 Processor -- start 2024-04-17T06:10:08.029 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.030+0000 7f3b0fa81640 1 -- start start 2024-04-17T06:10:08.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0fa81640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b0809af30 0x7f3b080cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0fa81640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080d1cf0 0x7f3b080cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0fa81640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b08162190 0x7f3b0814bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0fa81640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080cd2a0 con 0x7f3b0809af30 2024-04-17T06:10:08.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0fa81640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3b080ccfc0 con 0x7f3b080d1cf0 2024-04-17T06:10:08.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0fa81640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3b08162570 con 0x7f3b08162190 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b0809af30 0x7f3b080cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b0809af30 0x7f3b080cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55806/0 (socket says 172.21.15.135:55806) 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 -- 172.21.15.135:0/148543179 learned_addr learned my addr 172.21.15.135:0/148543179 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0e01e640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b08162190 0x7f3b0814bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d01c640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080d1cf0 0x7f3b080cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 -- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b08162190 msgr2=0x7f3b0814bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b08162190 0x7f3b0814bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 -- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080d1cf0 msgr2=0x7f3b080cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080d1cf0 0x7f3b080cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 -- 172.21.15.135:0/148543179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b080cb2a0 con 0x7f3b0809af30 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0e01e640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b08162190 0x7f3b0814bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d01c640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080d1cf0 0x7f3b080cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:08.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0d81d640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b0809af30 0x7f3b080cc5c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400d620 tx=0x7f3b0400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3afe7fc640 1 -- 172.21.15.135:0/148543179 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400b620 con 0x7f3b0809af30 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3afe7fc640 1 -- 172.21.15.135:0/148543179 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b04019070 con 0x7f3b0809af30 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3afe7fc640 1 -- 172.21.15.135:0/148543179 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400ba00 con 0x7f3b0809af30 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b080cb520 con 0x7f3b0809af30 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.031+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b080cbae0 con 0x7f3b0809af30 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.032+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b08074940 con 0x7f3b0809af30 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.033+0000 7f3afe7fc640 1 -- 172.21.15.135:0/148543179 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b0400bba0 con 0x7f3b0809af30 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.033+0000 7f3afe7fc640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3aec06c8f0 0x7f3aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.033+0000 7f3afe7fc640 1 -- 172.21.15.135:0/148543179 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f3b0408c380 con 0x7f3b0809af30 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.033+0000 7f3b0d01c640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3aec06c8f0 0x7f3aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.034+0000 7f3b0d01c640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3aec06c8f0 0x7f3aec06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3af8036000 tx=0x7f3af8008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.037+0000 7f3afe7fc640 1 -- 172.21.15.135:0/148543179 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b080cb2a0 con 0x7f3b0809af30 2024-04-17T06:10:08.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.214+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3b080d2490 con 0x7f3b0809af30 2024-04-17T06:10:08.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.214+0000 7f3afe7fc640 1 -- 172.21.15.135:0/148543179 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f3b0405a3b0 con 0x7f3b0809af30 2024-04-17T06:10:08.213 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3aec06c8f0 msgr2=0x7f3aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3aec06c8f0 0x7f3aec06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3af8036000 tx=0x7f3af8008030 comp rx=0 tx=0).stop 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b0809af30 msgr2=0x7f3b080cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b0809af30 0x7f3b080cc5c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3b0400d620 tx=0x7f3b0400dae0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 shutdown_connections 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3b08162190 0x7f3b0814bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3b080d1cf0 0x7f3b080cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3aec06c8f0 0x7f3aec06edb0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3af8036000 tx=0x7f3af8008030 comp rx=0 tx=0).stop 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 --2- 172.21.15.135:0/148543179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b0809af30 0x7f3b080cc5c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 >> 172.21.15.135:0/148543179 conn(0x7f3b080b41a0 msgr2=0x7f3b080ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.216+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 shutdown_connections 2024-04-17T06:10:08.215 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.217+0000 7f3b0fa81640 1 -- 172.21.15.135:0/148543179 wait complete. 2024-04-17T06:10:08.225 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T06:10:08.329 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.331+0000 7f23ab55f640 1 Processor -- start 2024-04-17T06:10:08.330 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.332+0000 7f23ab55f640 1 -- start start 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.332+0000 7f23ab55f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a415ec00 0x7f23a40d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.332+0000 7f23ab55f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23a40d43d0 0x7f23a40d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.332+0000 7f23ab55f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f23a40d1810 0x7f23a40730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.332+0000 7f23ab55f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f23a415efe0 con 0x7f23a40d1810 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.332+0000 7f23ab55f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f23a40d3f40 con 0x7f23a40d43d0 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.332+0000 7f23ab55f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f23a40d1c70 con 0x7f23a415ec00 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a9afc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f23a40d1810 0x7f23a40730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a9afc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f23a40d1810 0x7f23a40730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55820/0 (socket says 172.21.15.135:55820) 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a9afc640 1 -- 172.21.15.135:0/2200133535 learned_addr learned my addr 172.21.15.135:0/2200133535 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a92fb640 1 --2- 172.21.15.135:0/2200133535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a415ec00 0x7f23a40d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a8afa640 1 --2- 172.21.15.135:0/2200133535 >> v2:172.21.15.190:3300/0 conn(0x7f23a40d43d0 0x7f23a40d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a9afc640 1 -- 172.21.15.135:0/2200133535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a415ec00 msgr2=0x7f23a40d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:08.332 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a9afc640 1 --2- 172.21.15.135:0/2200133535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a415ec00 0x7f23a40d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a9afc640 1 -- 172.21.15.135:0/2200133535 >> v2:172.21.15.190:3300/0 conn(0x7f23a40d43d0 msgr2=0x7f23a40d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:08.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a9afc640 1 --2- 172.21.15.135:0/2200133535 >> v2:172.21.15.190:3300/0 conn(0x7f23a40d43d0 0x7f23a40d12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.333+0000 7f23a9afc640 1 -- 172.21.15.135:0/2200133535 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a40d3330 con 0x7f23a40d1810 2024-04-17T06:10:08.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23a9afc640 1 --2- 172.21.15.135:0/2200133535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a40d1810 0x7f23a40730c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f23a40741e0 tx=0x7f239c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3756cf36eb21c6f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f2397fff640 1 -- 172.21.15.135:0/2200133535 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c02cb00 con 0x7f23a40d1810 2024-04-17T06:10:08.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f2397fff640 1 -- 172.21.15.135:0/2200133535 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c02cca0 con 0x7f23a40d1810 2024-04-17T06:10:08.333 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f2397fff640 1 -- 172.21.15.135:0/2200133535 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c034610 con 0x7f23a40d1810 2024-04-17T06:10:08.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 -- 172.21.15.135:0/2200133535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a40d1810 msgr2=0x7f23a40730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.336 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 --2- 172.21.15.135:0/2200133535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a40d1810 0x7f23a40730c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f23a40741e0 tx=0x7f239c02bf60 comp rx=0 tx=0).stop 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 -- 172.21.15.135:0/2200133535 shutdown_connections 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 --2- 172.21.15.135:0/2200133535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a40d1810 0x7f23a40730c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 --2- 172.21.15.135:0/2200133535 >> v2:172.21.15.190:3300/0 conn(0x7f23a40d43d0 0x7f23a40d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 --2- 172.21.15.135:0/2200133535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a415ec00 0x7f23a40d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 -- 172.21.15.135:0/2200133535 >> 172.21.15.135:0/2200133535 conn(0x7f23a406cc90 msgr2=0x7f23a406d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 -- 172.21.15.135:0/2200133535 shutdown_connections 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.334+0000 7f23ab55f640 1 -- 172.21.15.135:0/2200133535 wait complete. 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23ab55f640 1 Processor -- start 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23ab55f640 1 -- start start 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23ab55f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a40d1810 0x7f23a4150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23ab55f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a40d43d0 0x7f23a40cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23ab55f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a415ec00 0x7f23a41513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23ab55f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23a40d3bd0 con 0x7f23a415ec00 2024-04-17T06:10:08.337 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23ab55f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f23a40d38f0 con 0x7f23a40d43d0 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23ab55f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f23a40d2ed0 con 0x7f23a40d1810 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23a9afc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a415ec00 0x7f23a41513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23a9afc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a415ec00 0x7f23a41513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55828/0 (socket says 172.21.15.135:55828) 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.336+0000 7f23a9afc640 1 -- 172.21.15.135:0/588655059 learned_addr learned my addr 172.21.15.135:0/588655059 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23a92fb640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a40d1810 0x7f23a4150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23a8afa640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a40d43d0 0x7f23a40cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23a9afc640 1 -- 172.21.15.135:0/588655059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a40d1810 msgr2=0x7f23a4150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23a9afc640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a40d1810 0x7f23a4150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23a9afc640 1 -- 172.21.15.135:0/588655059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a40d43d0 msgr2=0x7f23a40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23a9afc640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a40d43d0 0x7f23a40cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23a9afc640 1 -- 172.21.15.135:0/588655059 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a40cc3f0 con 0x7f23a415ec00 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23a9afc640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a415ec00 0x7f23a41513d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f239c036040 tx=0x7f239c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.338 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f2395ffb640 1 -- 172.21.15.135:0/588655059 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c043020 con 0x7f23a415ec00 2024-04-17T06:10:08.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f2395ffb640 1 -- 172.21.15.135:0/588655059 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239c02ce50 con 0x7f23a415ec00 2024-04-17T06:10:08.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a40cc6b0 con 0x7f23a415ec00 2024-04-17T06:10:08.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f2395ffb640 1 -- 172.21.15.135:0/588655059 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c009510 con 0x7f23a415ec00 2024-04-17T06:10:08.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.337+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a40ccf50 con 0x7f23a415ec00 2024-04-17T06:10:08.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.338+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a40cc880 con 0x7f23a415ec00 2024-04-17T06:10:08.339 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.339+0000 7f2395ffb640 1 -- 172.21.15.135:0/588655059 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f239c046020 con 0x7f23a415ec00 2024-04-17T06:10:08.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.339+0000 7f2395ffb640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f238806c920 0x7f238806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.339+0000 7f2395ffb640 1 -- 172.21.15.135:0/588655059 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f239c0ae6b0 con 0x7f23a415ec00 2024-04-17T06:10:08.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.339+0000 7f23a92fb640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f238806c920 0x7f238806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.340 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.340+0000 7f23a92fb640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f238806c920 0x7f238806ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f23a0006fd0 tx=0x7f23a0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.341 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.343+0000 7f2395ffb640 1 -- 172.21.15.135:0/588655059 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f239c04d030 con 0x7f23a415ec00 2024-04-17T06:10:08.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.530+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23a4074940 con 0x7f23a415ec00 2024-04-17T06:10:08.528 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.530+0000 7f2395ffb640 1 -- 172.21.15.135:0/588655059 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f239c07c650 con 0x7f23a415ec00 2024-04-17T06:10:08.528 INFO:teuthology.orchestra.run.smithi135.stderr:listed 0 entries 2024-04-17T06:10:08.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f238806c920 msgr2=0x7f238806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f238806c920 0x7f238806ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f23a0006fd0 tx=0x7f23a0008040 comp rx=0 tx=0).stop 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a415ec00 msgr2=0x7f23a41513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a415ec00 0x7f23a41513d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f239c036040 tx=0x7f239c004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 shutdown_connections 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f238806c920 0x7f238806ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a415ec00 0x7f23a41513d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23a40d43d0 0x7f23a40cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 --2- 172.21.15.135:0/588655059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23a40d1810 0x7f23a4150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 >> 172.21.15.135:0/588655059 conn(0x7f23a406cc90 msgr2=0x7f23a40b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 shutdown_connections 2024-04-17T06:10:08.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.532+0000 7f23ab55f640 1 -- 172.21.15.135:0/588655059 wait complete. 2024-04-17T06:10:08.541 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-17T06:10:08.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:08.541 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-17T06:10:08.577 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T06:10:08.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:08.577 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-17T06:10:08.654 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:08.654 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:08.670 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:08.670 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:08.726 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:08.727 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:08.783 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:08.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.927+0000 7f1f93ac8640 1 Processor -- start 2024-04-17T06:10:08.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.927+0000 7f1f93ac8640 1 -- start start 2024-04-17T06:10:08.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f93ac8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 0x7f1f8c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f93ac8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1f8c0cd630 0x7f1f8c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f93ac8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c06db80 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f93ac8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1f8c06de60 con 0x7f1f8c0cd630 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f93ac8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c0cce30 con 0x7f1f8c15ee30 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f92065640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1f8c0cd630 0x7f1f8c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f92065640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1f8c0cd630 0x7f1f8c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55832/0 (socket says 172.21.15.135:55832) 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f92065640 1 -- 172.21.15.135:0/1981866752 learned_addr learned my addr 172.21.15.135:0/1981866752 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.928+0000 7f1f91864640 1 --2- 172.21.15.135:0/1981866752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 0x7f1f8c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.929+0000 7f1f92065640 1 -- 172.21.15.135:0/1981866752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 msgr2=0x7f1f8c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.929+0000 7f1f92065640 1 --2- 172.21.15.135:0/1981866752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 0x7f1f8c09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.929+0000 7f1f92065640 1 -- 172.21.15.135:0/1981866752 >> v1:172.21.15.190:6789/0 conn(0x7f1f8c09bbe0 legacy=0x7f1f8c09af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.929+0000 7f1f92065640 1 -- 172.21.15.135:0/1981866752 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c0cd110 con 0x7f1f8c0cd630 2024-04-17T06:10:08.928 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.929+0000 7f1f91864640 1 --2- 172.21.15.135:0/1981866752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 0x7f1f8c09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:08.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.929+0000 7f1f92065640 1 --2- 172.21.15.135:0/1981866752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c0cd630 0x7f1f8c0cda80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009e50 tx=0x7f1f8402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d427e741c1c40b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f90862640 1 -- 172.21.15.135:0/1981866752 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8402cb00 con 0x7f1f8c0cd630 2024-04-17T06:10:08.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f90862640 1 -- 172.21.15.135:0/1981866752 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8402cca0 con 0x7f1f8c0cd630 2024-04-17T06:10:08.929 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f90862640 1 -- 172.21.15.135:0/1981866752 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84034610 con 0x7f1f8c0cd630 2024-04-17T06:10:08.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1981866752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c0cd630 msgr2=0x7f1f8c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1981866752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c0cd630 0x7f1f8c0cda80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009e50 tx=0x7f1f8402bf60 comp rx=0 tx=0).stop 2024-04-17T06:10:08.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1981866752 shutdown_connections 2024-04-17T06:10:08.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1981866752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c0cd630 0x7f1f8c0cda80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1981866752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 0x7f1f8c09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1981866752 >> 172.21.15.135:0/1981866752 conn(0x7f1f8c0b41a0 msgr2=0x7f1f8c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1981866752 shutdown_connections 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.930+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1981866752 wait complete. 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.931+0000 7f1f93ac8640 1 Processor -- start 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.931+0000 7f1f93ac8640 1 -- start start 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f93ac8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c09bbe0 0x7f1f8c0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f93ac8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f8c0cd630 0x7f1f8c14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f93ac8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 0x7f1f8c161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f93ac8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c06d910 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f93ac8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c06d790 con 0x7f1f8c0cd630 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f93ac8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c09b640 con 0x7f1f8c15ee30 2024-04-17T06:10:08.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f91864640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c09bbe0 0x7f1f8c0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f91864640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c09bbe0 0x7f1f8c0b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55842/0 (socket says 172.21.15.135:55842) 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f91864640 1 -- 172.21.15.135:0/1867698377 learned_addr learned my addr 172.21.15.135:0/1867698377 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f91864640 1 -- 172.21.15.135:0/1867698377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 msgr2=0x7f1f8c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f91864640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 0x7f1f8c161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f91864640 1 -- 172.21.15.135:0/1867698377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f8c0cd630 msgr2=0x7f1f8c14da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f91864640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f8c0cd630 0x7f1f8c14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.932+0000 7f1f91864640 1 -- 172.21.15.135:0/1867698377 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c0c6380 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.933+0000 7f1f91864640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c09bbe0 0x7f1f8c0b53f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800d800 tx=0x7f1f8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.933+0000 7f1f827fc640 1 -- 172.21.15.135:0/1867698377 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88004030 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.933+0000 7f1f827fc640 1 -- 172.21.15.135:0/1867698377 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f880041d0 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.933+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c0c6660 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.933+0000 7f1f827fc640 1 -- 172.21.15.135:0/1867698377 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f880096a0 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.933+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c156ee0 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.934+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8c074940 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.934+0000 7f1f827fc640 1 -- 172.21.15.135:0/1867698377 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f88009840 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.934+0000 7f1f827fc640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f7006c8f0 0x7f1f7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:08.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.935+0000 7f1f827fc640 1 -- 172.21.15.135:0/1867698377 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1f8808c2e0 con 0x7f1f8c09bbe0 2024-04-17T06:10:08.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.935+0000 7f1f91063640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f7006c8f0 0x7f1f7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:08.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.935+0000 7f1f91063640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f7006c8f0 0x7f1f7006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000f60 tx=0x7f1f7c0079d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:08.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:08.939+0000 7f1f827fc640 1 -- 172.21.15.135:0/1867698377 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f8805a280 con 0x7f1f8c09bbe0 2024-04-17T06:10:09.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.114+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f8c0b38e0 con 0x7f1f8c09bbe0 2024-04-17T06:10:09.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.116+0000 7f1f827fc640 1 -- 172.21.15.135:0/1867698377 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18950 (secure 0 0 0) 0x7f1f88059b80 con 0x7f1f8c09bbe0 2024-04-17T06:10:09.114 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:09.114 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:01.008924+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:09.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f7006c8f0 msgr2=0x7f1f7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f7006c8f0 0x7f1f7006edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000f60 tx=0x7f1f7c0079d0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c09bbe0 msgr2=0x7f1f8c0b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c09bbe0 0x7f1f8c0b53f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800d800 tx=0x7f1f8800dcc0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 shutdown_connections 2024-04-17T06:10:09.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f8c15ee30 0x7f1f8c161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f8c0cd630 0x7f1f8c14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f7006c8f0 0x7f1f7006edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 --2- 172.21.15.135:0/1867698377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c09bbe0 0x7f1f8c0b53f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 >> 172.21.15.135:0/1867698377 conn(0x7f1f8c0b41a0 msgr2=0x7f1f8c0705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:09.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 shutdown_connections 2024-04-17T06:10:09.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.118+0000 7f1f93ac8640 1 -- 172.21.15.135:0/1867698377 wait complete. 2024-04-17T06:10:09.127 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:10:09.128 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:10:09.233 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.235+0000 7f496799d640 1 Processor -- start 2024-04-17T06:10:09.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.236+0000 7f496799d640 1 -- start start 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.236+0000 7f496799d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49600d43d0 0x7f49600d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.236+0000 7f496799d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49600d30a0 con 0x7f49600d1810 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.236+0000 7f496799d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49600d3330 con 0x7f496015ec00 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.236+0000 7f496799d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f496015efe0 con 0x7f49600d43d0 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.237+0000 7f4964f38640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f49600d43d0 0x7f49600d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.237+0000 7f4965f3a640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f49600d1810 0x7f49600b7a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42336/0 (socket says 172.21.15.135:42336) 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.237+0000 7f4965f3a640 1 -- 172.21.15.135:0/1991033906 learned_addr learned my addr 172.21.15.135:0/1991033906 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.237+0000 7f4964f38640 1 -- 172.21.15.135:0/1991033906 >> v1:172.21.15.190:6789/0 conn(0x7f496015ec00 legacy=0x7f49600d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.237+0000 7f4964f38640 1 -- 172.21.15.135:0/1991033906 >> v1:172.21.15.135:6789/0 conn(0x7f49600d1810 legacy=0x7f49600b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.237+0000 7f495ffff640 1 -- 172.21.15.135:0/1991033906 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3385871656 0 0) 0x7f49600d30a0 con 0x7f49600d1810 2024-04-17T06:10:09.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.237+0000 7f4964f38640 1 -- 172.21.15.135:0/1991033906 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49600d3610 con 0x7f49600d43d0 2024-04-17T06:10:09.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f4964f38640 1 --2- 172.21.15.135:0/1991033906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f49600d12d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4954009d90 tx=0x7f495402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e40052e41290e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f495ffff640 1 -- 172.21.15.135:0/1991033906 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954039070 con 0x7f49600d43d0 2024-04-17T06:10:09.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f495ffff640 1 -- 172.21.15.135:0/1991033906 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f495402cd80 con 0x7f49600d43d0 2024-04-17T06:10:09.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f495ffff640 1 -- 172.21.15.135:0/1991033906 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954034660 con 0x7f49600d43d0 2024-04-17T06:10:09.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f496799d640 1 -- 172.21.15.135:0/1991033906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 msgr2=0x7f49600d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f496799d640 1 --2- 172.21.15.135:0/1991033906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f49600d12d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4954009d90 tx=0x7f495402bfd0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f496799d640 1 -- 172.21.15.135:0/1991033906 shutdown_connections 2024-04-17T06:10:09.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f496799d640 1 --2- 172.21.15.135:0/1991033906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f49600d12d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.238+0000 7f496799d640 1 -- 172.21.15.135:0/1991033906 >> 172.21.15.135:0/1991033906 conn(0x7f496006cc90 msgr2=0x7f496006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:09.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.239+0000 7f496799d640 1 -- 172.21.15.135:0/1991033906 shutdown_connections 2024-04-17T06:10:09.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.239+0000 7f496799d640 1 -- 172.21.15.135:0/1991033906 wait complete. 2024-04-17T06:10:09.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.240+0000 7f496799d640 1 Processor -- start 2024-04-17T06:10:09.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.240+0000 7f496799d640 1 -- start start 2024-04-17T06:10:09.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f496799d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49600d1810 0x7f49600c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f496799d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f4960150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f496799d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f496015ec00 0x7f49600cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f496799d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49600d1c60 con 0x7f49600d43d0 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f496799d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f49600d3f40 con 0x7f49600d1810 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f496799d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f49600d3bd0 con 0x7f496015ec00 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f4960150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f4960150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55856/0 (socket says 172.21.15.135:55856) 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 -- 172.21.15.135:0/3196126293 learned_addr learned my addr 172.21.15.135:0/3196126293 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4965739640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49600d1810 0x7f49600c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4965f3a640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f496015ec00 0x7f49600cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 -- 172.21.15.135:0/3196126293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f496015ec00 msgr2=0x7f49600cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f496015ec00 0x7f49600cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 -- 172.21.15.135:0/3196126293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49600d1810 msgr2=0x7f49600c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49600d1810 0x7f49600c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 -- 172.21.15.135:0/3196126293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49600ce250 con 0x7f49600d43d0 2024-04-17T06:10:09.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f4964f38640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f4960150f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4954037040 tx=0x7f4954004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f495dffb640 1 -- 172.21.15.135:0/3196126293 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954039050 con 0x7f49600d43d0 2024-04-17T06:10:09.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f495dffb640 1 -- 172.21.15.135:0/3196126293 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f495403a040 con 0x7f49600d43d0 2024-04-17T06:10:09.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49600ce530 con 0x7f49600d43d0 2024-04-17T06:10:09.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.241+0000 7f495dffb640 1 -- 172.21.15.135:0/3196126293 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954003bd0 con 0x7f49600d43d0 2024-04-17T06:10:09.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.242+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4960151440 con 0x7f49600d43d0 2024-04-17T06:10:09.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.243+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4960074940 con 0x7f49600d43d0 2024-04-17T06:10:09.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.243+0000 7f495dffb640 1 -- 172.21.15.135:0/3196126293 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4954047030 con 0x7f49600d43d0 2024-04-17T06:10:09.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.244+0000 7f495dffb640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f494c06c8f0 0x7f494c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.244+0000 7f495dffb640 1 -- 172.21.15.135:0/3196126293 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f49540ae360 con 0x7f49600d43d0 2024-04-17T06:10:09.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.244+0000 7f4965739640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f494c06c8f0 0x7f494c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.244+0000 7f4965739640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f494c06c8f0 0x7f494c06edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4958000f60 tx=0x7f4958009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.248+0000 7f495dffb640 1 -- 172.21.15.135:0/3196126293 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f495404e020 con 0x7f49600d43d0 2024-04-17T06:10:09.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.421+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49600b9d20 con 0x7f49600d43d0 2024-04-17T06:10:09.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.422+0000 7f495dffb640 1 -- 172.21.15.135:0/3196126293 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f495407bdd0 con 0x7f49600d43d0 2024-04-17T06:10:09.420 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.423+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f494c06c8f0 msgr2=0x7f494c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f494c06c8f0 0x7f494c06edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4958000f60 tx=0x7f4958009040 comp rx=0 tx=0).stop 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 msgr2=0x7f4960150f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f4960150f00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4954037040 tx=0x7f4954004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 shutdown_connections 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f496015ec00 0x7f49600cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f494c06c8f0 0x7f494c06edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49600d43d0 0x7f4960150f00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.422 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 --2- 172.21.15.135:0/3196126293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49600d1810 0x7f49600c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 >> 172.21.15.135:0/3196126293 conn(0x7f496006cc90 msgr2=0x7f49600b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:09.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 shutdown_connections 2024-04-17T06:10:09.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.424+0000 7f496799d640 1 -- 172.21.15.135:0/3196126293 wait complete. 2024-04-17T06:10:09.433 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:10:09.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.541+0000 7f6e4ef96640 1 Processor -- start 2024-04-17T06:10:09.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.541+0000 7f6e4ef96640 1 -- start start 2024-04-17T06:10:09.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e4ef96640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4815cac0 0x7f6e4806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e4ef96640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6e4806d020 0x7f6e48070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e4ef96640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e4815cea0 con 0x7f6e480d03d0 2024-04-17T06:10:09.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e4ef96640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6e480d0940 con 0x7f6e4806d020 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e4ef96640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6e4806c540 con 0x7f6e4815cac0 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e47fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6e4806d020 0x7f6e48070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e47fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6e4806d020 0x7f6e48070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55866/0 (socket says 172.21.15.135:55866) 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e4d533640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f6e480d03d0 0x7f6e480cec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42340/0 (socket says 172.21.15.135:42340) 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e47fff640 1 -- 172.21.15.135:0/109526117 learned_addr learned my addr 172.21.15.135:0/109526117 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.542+0000 7f6e4cd32640 1 --2- 172.21.15.135:0/109526117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4815cac0 0x7f6e4806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e47fff640 1 -- 172.21.15.135:0/109526117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4815cac0 msgr2=0x7f6e4806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e47fff640 1 --2- 172.21.15.135:0/109526117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4815cac0 0x7f6e4806e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e47fff640 1 -- 172.21.15.135:0/109526117 >> v1:172.21.15.135:6789/0 conn(0x7f6e480d03d0 legacy=0x7f6e480cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e47fff640 1 -- 172.21.15.135:0/109526117 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4806c820 con 0x7f6e4806d020 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e477fe640 1 -- 172.21.15.135:0/109526117 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708739568 0 0) 0x7f6e4815cea0 con 0x7f6e480d03d0 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e47fff640 1 --2- 172.21.15.135:0/109526117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4806d020 0x7f6e48070370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009870 tx=0x7f6e3c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ef056746ddfc2b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e477fe640 1 -- 172.21.15.135:0/109526117 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c034070 con 0x7f6e4806d020 2024-04-17T06:10:09.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e477fe640 1 -- 172.21.15.135:0/109526117 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3c030840 con 0x7f6e4806d020 2024-04-17T06:10:09.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.543+0000 7f6e477fe640 1 -- 172.21.15.135:0/109526117 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c030c10 con 0x7f6e4806d020 2024-04-17T06:10:09.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.544+0000 7f6e4ef96640 1 -- 172.21.15.135:0/109526117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4806d020 msgr2=0x7f6e48070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.544+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/109526117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4806d020 0x7f6e48070370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009870 tx=0x7f6e3c02b980 comp rx=0 tx=0).stop 2024-04-17T06:10:09.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.544+0000 7f6e4ef96640 1 -- 172.21.15.135:0/109526117 shutdown_connections 2024-04-17T06:10:09.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.544+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/109526117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4806d020 0x7f6e48070370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.544+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/109526117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4815cac0 0x7f6e4806e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.544+0000 7f6e4ef96640 1 -- 172.21.15.135:0/109526117 >> 172.21.15.135:0/109526117 conn(0x7f6e480cd7e0 msgr2=0x7f6e480cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.544+0000 7f6e4ef96640 1 -- 172.21.15.135:0/109526117 shutdown_connections 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.544+0000 7f6e4ef96640 1 -- 172.21.15.135:0/109526117 wait complete. 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.545+0000 7f6e4ef96640 1 Processor -- start 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.545+0000 7f6e4ef96640 1 -- start start 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4ef96640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4806d020 0x7f6e4809c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4ef96640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e480d03d0 0x7f6e4814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4ef96640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4815cac0 0x7f6e4809ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4ef96640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4806ef10 con 0x7f6e4815cac0 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4ef96640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4806ed90 con 0x7f6e480d03d0 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4ef96640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6e480708b0 con 0x7f6e4806d020 2024-04-17T06:10:09.546 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4d533640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4815cac0 0x7f6e4809ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4d533640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4815cac0 0x7f6e4809ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55872/0 (socket says 172.21.15.135:55872) 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.546+0000 7f6e4d533640 1 -- 172.21.15.135:0/791353489 learned_addr learned my addr 172.21.15.135:0/791353489 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4cd32640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4806d020 0x7f6e4809c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e47fff640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e480d03d0 0x7f6e4814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4d533640 1 -- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4806d020 msgr2=0x7f6e4809c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4d533640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4806d020 0x7f6e4809c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4d533640 1 -- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e480d03d0 msgr2=0x7f6e4814c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4d533640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e480d03d0 0x7f6e4814c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4d533640 1 -- 172.21.15.135:0/791353489 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4809cd50 con 0x7f6e4815cac0 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4cd32640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4806d020 0x7f6e4809c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4d533640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4815cac0 0x7f6e4809ddf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c7a0 tx=0x7f6e4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e457fa640 1 -- 172.21.15.135:0/791353489 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40018070 con 0x7f6e4815cac0 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e457fa640 1 -- 172.21.15.135:0/791353489 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e4000a640 con 0x7f6e4815cac0 2024-04-17T06:10:09.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e457fa640 1 -- 172.21.15.135:0/791353489 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000aab0 con 0x7f6e4815cac0 2024-04-17T06:10:09.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4809d030 con 0x7f6e4815cac0 2024-04-17T06:10:09.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.547+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4809d5f0 con 0x7f6e4815cac0 2024-04-17T06:10:09.548 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.548+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e48074940 con 0x7f6e4815cac0 2024-04-17T06:10:09.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.548+0000 7f6e457fa640 1 -- 172.21.15.135:0/791353489 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e4001b070 con 0x7f6e4815cac0 2024-04-17T06:10:09.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.549+0000 7f6e457fa640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e3406c8f0 0x7f6e3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.549+0000 7f6e457fa640 1 -- 172.21.15.135:0/791353489 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f6e4008b580 con 0x7f6e4815cac0 2024-04-17T06:10:09.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.549+0000 7f6e4cd32640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e3406c8f0 0x7f6e3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.549 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.549+0000 7f6e4cd32640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e3406c8f0 0x7f6e3406edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6e38006fd0 tx=0x7f6e38008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.551 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.553+0000 7f6e457fa640 1 -- 172.21.15.135:0/791353489 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6e400595b0 con 0x7f6e4815cac0 2024-04-17T06:10:09.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.728+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6e48160250 con 0x7f6e4815cac0 2024-04-17T06:10:09.727 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.729+0000 7f6e457fa640 1 -- 172.21.15.135:0/791353489 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f6e40058eb0 con 0x7f6e4815cac0 2024-04-17T06:10:09.727 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:09.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e3406c8f0 msgr2=0x7f6e3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e3406c8f0 0x7f6e3406edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6e38006fd0 tx=0x7f6e38008040 comp rx=0 tx=0).stop 2024-04-17T06:10:09.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4815cac0 msgr2=0x7f6e4809ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4815cac0 0x7f6e4809ddf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c7a0 tx=0x7f6e4000cc60 comp rx=0 tx=0).stop 2024-04-17T06:10:09.729 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 shutdown_connections 2024-04-17T06:10:09.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6e3406c8f0 0x7f6e3406edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e4815cac0 0x7f6e4809ddf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e480d03d0 0x7f6e4814c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 --2- 172.21.15.135:0/791353489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4806d020 0x7f6e4809c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 >> 172.21.15.135:0/791353489 conn(0x7f6e480cd7e0 msgr2=0x7f6e4806b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:09.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 shutdown_connections 2024-04-17T06:10:09.730 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.731+0000 7f6e4ef96640 1 -- 172.21.15.135:0/791353489 wait complete. 2024-04-17T06:10:09.740 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:10:09.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.848+0000 7f1f712d3640 1 Processor -- start 2024-04-17T06:10:09.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.848+0000 7f1f712d3640 1 -- start start 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.849+0000 7f1f712d3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f6c09bd50 0x7f1f6c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.849+0000 7f1f712d3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f6c09af30 0x7f1f6c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.849+0000 7f1f712d3640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6c0d13f0 con 0x7f1f6c15ed00 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.849+0000 7f1f712d3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f1f6c0d19b0 con 0x7f1f6c09af30 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.849+0000 7f1f712d3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c0d16d0 con 0x7f1f6c09bd50 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.849+0000 7f1f6affd640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f1f6c15ed00 0x7f1f6c0d0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42342/0 (socket says 172.21.15.135:42342) 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.849+0000 7f1f6affd640 1 -- 172.21.15.135:0/3403844493 learned_addr learned my addr 172.21.15.135:0/3403844493 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6a7fc640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f6c09bd50 0x7f1f6c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914076645 0 0) 0x7f1f6c0d13f0 con 0x7f1f6c15ed00 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6b7fe640 1 --2- 172.21.15.135:0/3403844493 >> v2:172.21.15.190:3300/0 conn(0x7f1f6c09af30 0x7f1f6c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f58003520 con 0x7f1f6c15ed00 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2167833022 0 0) 0x7f1f58003520 con 0x7f1f6c15ed00 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f6c09bd50 msgr2=0x7f1f6c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f6c09bd50 0x7f1f6c09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 >> v2:172.21.15.190:3300/0 conn(0x7f1f6c09af30 msgr2=0x7f1f6c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 --2- 172.21.15.135:0/3403844493 >> v2:172.21.15.190:3300/0 conn(0x7f1f6c09af30 0x7f1f6c06d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c06cc20 con 0x7f1f6c15ed00 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f1f5c0019f0 con 0x7f1f6c15ed00 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 >> v1:172.21.15.135:6789/0 conn(0x7f1f6c15ed00 legacy=0x7f1f6c0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f6c09bd50 0x7f1f580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f58003f30 0x7f1f58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58004d40 0x7f1f580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c0d19b0 con 0x7f1f58003f30 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c0d16d0 con 0x7f1f58004d40 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5800b9c0 con 0x7f1f6c09bd50 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6a7fc640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f58003f30 0x7f1f58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6affd640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f6c09bd50 0x7f1f580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6b7fe640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58004d40 0x7f1f580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6a7fc640 1 -- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f6c09bd50 msgr2=0x7f1f580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6a7fc640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f6c09bd50 0x7f1f580039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6a7fc640 1 -- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58004d40 msgr2=0x7f1f580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6a7fc640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58004d40 0x7f1f580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6a7fc640 1 -- 172.21.15.135:0/3403844493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f58003520 con 0x7f1f58003f30 2024-04-17T06:10:09.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.850+0000 7f1f6b7fe640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f58004d40 0x7f1f580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:09.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.851+0000 7f1f6a7fc640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f58003f30 0x7f1f58004800 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009ac0 tx=0x7f1f58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fdc981237bfac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.851+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54039070 con 0x7f1f58003f30 2024-04-17T06:10:09.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.851+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5402c6b0 con 0x7f1f58003f30 2024-04-17T06:10:09.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.851+0000 7f1f69ffb640 1 -- 172.21.15.135:0/3403844493 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54009690 con 0x7f1f58003f30 2024-04-17T06:10:09.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.852+0000 7f1f712d3640 1 -- 172.21.15.135:0/3403844493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f58003f30 msgr2=0x7f1f58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.852+0000 7f1f712d3640 1 --2- 172.21.15.135:0/3403844493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f58003f30 0x7f1f58004800 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009ac0 tx=0x7f1f58005830 comp rx=0 tx=0).stop 2024-04-17T06:10:09.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.852+0000 7f1f6b7fe640 1 -- 172.21.15.135:0/3403844493 reap_dead start 2024-04-17T06:10:09.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.852+0000 7f1f712d3640 1 -- 172.21.15.135:0/3403844493 shutdown_connections 2024-04-17T06:10:09.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.852+0000 7f1f712d3640 1 -- 172.21.15.135:0/3403844493 >> 172.21.15.135:0/3403844493 conn(0x7f1f6c0b4190 msgr2=0x7f1f6c0b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:09.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.852+0000 7f1f712d3640 1 -- 172.21.15.135:0/3403844493 shutdown_connections 2024-04-17T06:10:09.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.852+0000 7f1f712d3640 1 -- 172.21.15.135:0/3403844493 wait complete. 2024-04-17T06:10:09.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.853+0000 7f1f712d3640 1 Processor -- start 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.853+0000 7f1f712d3640 1 -- start start 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f712d3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58003f30 0x7f1f6c160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f712d3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6c0cb1d0 0x7f1f6c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f712d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6c151ea0 0x7f1f6c152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f712d3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c09ab70 con 0x7f1f6c151ea0 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f712d3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c09a9f0 con 0x7f1f6c0cb1d0 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f712d3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6c09acf0 con 0x7f1f58003f30 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6c151ea0 0x7f1f6c152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6c151ea0 0x7f1f6c152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55898/0 (socket says 172.21.15.135:55898) 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 -- 172.21.15.135:0/3223318615 learned_addr learned my addr 172.21.15.135:0/3223318615 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6affd640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58003f30 0x7f1f6c160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6a7fc640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6c0cb1d0 0x7f1f6c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 -- 172.21.15.135:0/3223318615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58003f30 msgr2=0x7f1f6c160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58003f30 0x7f1f6c160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 -- 172.21.15.135:0/3223318615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6c0cb1d0 msgr2=0x7f1f6c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6c0cb1d0 0x7f1f6c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 -- 172.21.15.135:0/3223318615 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c160f50 con 0x7f1f6c151ea0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.854+0000 7f1f6b7fe640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6c151ea0 0x7f1f6c152320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c06c570 tx=0x7f1f64011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.855+0000 7f1f637fe640 1 -- 172.21.15.135:0/3223318615 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64003610 con 0x7f1f6c151ea0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.855+0000 7f1f637fe640 1 -- 172.21.15.135:0/3223318615 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f6401b070 con 0x7f1f6c151ea0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.855+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6c161230 con 0x7f1f6c151ea0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.855+0000 7f1f637fe640 1 -- 172.21.15.135:0/3223318615 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64003a10 con 0x7f1f6c151ea0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.855+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6c1617f0 con 0x7f1f6c151ea0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.856+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6c068a20 con 0x7f1f6c151ea0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.856+0000 7f1f637fe640 1 -- 172.21.15.135:0/3223318615 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f64003bb0 con 0x7f1f6c151ea0 2024-04-17T06:10:09.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.857+0000 7f1f637fe640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f4c06c920 0x7f1f4c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:09.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.857+0000 7f1f637fe640 1 -- 172.21.15.135:0/3223318615 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1f6408c170 con 0x7f1f6c151ea0 2024-04-17T06:10:09.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.857+0000 7f1f6affd640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f4c06c920 0x7f1f4c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:09.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.857+0000 7f1f6affd640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f4c06c920 0x7f1f4c06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c002230 tx=0x7f1f5c006040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:09.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:09.861+0000 7f1f637fe640 1 -- 172.21.15.135:0/3223318615 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f6405a1a0 con 0x7f1f6c151ea0 2024-04-17T06:10:10.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.039+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f6c152860 con 0x7f1f6c151ea0 2024-04-17T06:10:10.037 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.039+0000 7f1f637fe640 1 -- 172.21.15.135:0/3223318615 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f1f64059aa0 con 0x7f1f6c151ea0 2024-04-17T06:10:10.037 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:10:10.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f4c06c920 msgr2=0x7f1f4c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f4c06c920 0x7f1f4c06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c002230 tx=0x7f1f5c006040 comp rx=0 tx=0).stop 2024-04-17T06:10:10.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6c151ea0 msgr2=0x7f1f6c152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6c151ea0 0x7f1f6c152320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c06c570 tx=0x7f1f64011860 comp rx=0 tx=0).stop 2024-04-17T06:10:10.039 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 shutdown_connections 2024-04-17T06:10:10.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1f4c06c920 0x7f1f4c06ede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f6c151ea0 0x7f1f6c152320 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f6c0cb1d0 0x7f1f6c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 --2- 172.21.15.135:0/3223318615 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f58003f30 0x7f1f6c160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 >> 172.21.15.135:0/3223318615 conn(0x7f1f6c0b4190 msgr2=0x7f1f6c070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:10.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 shutdown_connections 2024-04-17T06:10:10.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.041+0000 7f1f712d3640 1 -- 172.21.15.135:0/3223318615 wait complete. 2024-04-17T06:10:10.051 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:10:10.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.159+0000 7fd4d13ca640 1 Processor -- start 2024-04-17T06:10:10.157 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.159+0000 7fd4d13ca640 1 -- start start 2024-04-17T06:10:10.158 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4d13ca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc0d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4d13ca640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4cc0d19e0 0x7fd4cc0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4d13ca640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4cc0d3330 con 0x7fd4cc0d43d0 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4d13ca640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc15efe0 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4d13ca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4cc0d3f40 con 0x7fd4cc15ec00 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4caffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc0d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4cb7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4cc0d19e0 0x7fd4cc0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4caffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc0d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:50982/0 (socket says 172.21.15.135:50982) 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4ca7fc640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fd4cc0d43d0 0x7fd4cc0d12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:48180/0 (socket says 172.21.15.135:48180) 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.160+0000 7fd4caffd640 1 -- 172.21.15.135:0/3045898688 learned_addr learned my addr 172.21.15.135:0/3045898688 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.161+0000 7fd4caffd640 1 -- 172.21.15.135:0/3045898688 >> v2:172.21.15.190:3300/0 conn(0x7fd4cc0d19e0 msgr2=0x7fd4cc0b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.161+0000 7fd4caffd640 1 --2- 172.21.15.135:0/3045898688 >> v2:172.21.15.190:3300/0 conn(0x7fd4cc0d19e0 0x7fd4cc0b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.161+0000 7fd4caffd640 1 -- 172.21.15.135:0/3045898688 >> v1:172.21.15.190:6789/0 conn(0x7fd4cc0d43d0 legacy=0x7fd4cc0d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.159 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.161+0000 7fd4caffd640 1 -- 172.21.15.135:0/3045898688 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc0d3610 con 0x7fd4cc15ec00 2024-04-17T06:10:10.161 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.161+0000 7fd4caffd640 1 --2- 172.21.15.135:0/3045898688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc0d2990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009870 tx=0x7fd4bc02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa35f01ced7a122c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4c9ffb640 1 -- 172.21.15.135:0/3045898688 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc02f840 con 0x7fd4cc15ec00 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4c9ffb640 1 -- 172.21.15.135:0/3045898688 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc02f9e0 con 0x7fd4cc15ec00 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4c9ffb640 1 -- 172.21.15.135:0/3045898688 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc02fdb0 con 0x7fd4cc15ec00 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4d13ca640 1 -- 172.21.15.135:0/3045898688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 msgr2=0x7fd4cc0d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/3045898688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc0d2990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009870 tx=0x7fd4bc02ba20 comp rx=0 tx=0).stop 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4d13ca640 1 -- 172.21.15.135:0/3045898688 shutdown_connections 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/3045898688 >> v2:172.21.15.190:3300/0 conn(0x7fd4cc0d19e0 0x7fd4cc0b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/3045898688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc0d2990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4d13ca640 1 -- 172.21.15.135:0/3045898688 >> 172.21.15.135:0/3045898688 conn(0x7fd4cc06cc90 msgr2=0x7fd4cc06d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4d13ca640 1 -- 172.21.15.135:0/3045898688 shutdown_connections 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.162+0000 7fd4d13ca640 1 -- 172.21.15.135:0/3045898688 wait complete. 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4d13ca640 1 Processor -- start 2024-04-17T06:10:10.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4d13ca640 1 -- start start 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4d13ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc0d19e0 0x7fd4cc0cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4d13ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4cc0d43d0 0x7fd4cc1522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4d13ca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4d13ca640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0d2ed0 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4d13ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0d3bd0 con 0x7fd4cc0d43d0 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4d13ca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4cc0d3050 con 0x7fd4cc15ec00 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4caffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc0d19e0 0x7fd4cc0cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4caffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc0d19e0 0x7fd4cc0cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55908/0 (socket says 172.21.15.135:55908) 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.164+0000 7fd4caffd640 1 -- 172.21.15.135:0/2546033330 learned_addr learned my addr 172.21.15.135:0/2546033330 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4cb7fe640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4ca7fc640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4cc0d43d0 0x7fd4cc1522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4caffd640 1 -- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 msgr2=0x7fd4cc152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4caffd640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4caffd640 1 -- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4cc0d43d0 msgr2=0x7fd4cc1522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4caffd640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4cc0d43d0 0x7fd4cc1522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4caffd640 1 -- 172.21.15.135:0/2546033330 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc15f430 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4ca7fc640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4cc0d43d0 0x7fd4cc1522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4caffd640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc0d19e0 0x7fd4cc0cbc80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4b37fe640 1 -- 172.21.15.135:0/2546033330 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc037bc0 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4b37fe640 1 -- 172.21.15.135:0/2546033330 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc046070 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4b37fe640 1 -- 172.21.15.135:0/2546033330 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc041400 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc0ce980 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.165+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc15f9d0 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.166+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc074940 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.167+0000 7fd4b37fe640 1 -- 172.21.15.135:0/2546033330 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4bc040070 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.167+0000 7fd4b37fe640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4b806c8f0 0x7fd4b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.167+0000 7fd4b37fe640 1 -- 172.21.15.135:0/2546033330 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd4bc0ae4f0 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.167+0000 7fd4ca7fc640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4b806c8f0 0x7fd4b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.167 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.168+0000 7fd4ca7fc640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4b806c8f0 0x7fd4b806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0cb460 tx=0x7fd4c0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:10.169 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.172+0000 7fd4b37fe640 1 -- 172.21.15.135:0/2546033330 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4bc033030 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.344+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd4cc0b4e80 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.342 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.344+0000 7fd4b37fe640 1 -- 172.21.15.135:0/2546033330 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fd4bc07c5b0 con 0x7fd4cc0d19e0 2024-04-17T06:10:10.343 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:10.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.346+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4b806c8f0 msgr2=0x7fd4b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.346+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4b806c8f0 0x7fd4b806edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0cb460 tx=0x7fd4c0009040 comp rx=0 tx=0).stop 2024-04-17T06:10:10.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.346+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc0d19e0 msgr2=0x7fd4cc0cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.346+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc0d19e0 0x7fd4cc0cbc80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc000c00 tx=0x7fd4bc004730 comp rx=0 tx=0).stop 2024-04-17T06:10:10.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.347+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 shutdown_connections 2024-04-17T06:10:10.345 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.347+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4cc15ec00 0x7fd4cc152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.347+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4cc0d43d0 0x7fd4cc1522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.347+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd4b806c8f0 0x7fd4b806edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.347+0000 7fd4d13ca640 1 --2- 172.21.15.135:0/2546033330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4cc0d19e0 0x7fd4cc0cbc80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.347+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 >> 172.21.15.135:0/2546033330 conn(0x7fd4cc06cc90 msgr2=0x7fd4cc15db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:10.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.347+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 shutdown_connections 2024-04-17T06:10:10.346 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.347+0000 7fd4d13ca640 1 -- 172.21.15.135:0/2546033330 wait complete. 2024-04-17T06:10:10.356 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T06:10:10.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.464+0000 7fc2785d4640 1 Processor -- start 2024-04-17T06:10:10.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.464+0000 7fc2785d4640 1 -- start start 2024-04-17T06:10:10.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc2785d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015ff30 0x7fc27015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc2785d4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2700cfe50 0x7fc270072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc2785d4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc27006e170 con 0x7fc270071ec0 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc2785d4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc27006df20 con 0x7fc2700cfe50 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc2785d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc27006eaa0 con 0x7fc27015ff30 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc276b71640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2700cfe50 0x7fc270072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc276b71640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc2700cfe50 0x7fc270072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55924/0 (socket says 172.21.15.135:55924) 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc276b71640 1 -- 172.21.15.135:0/907952818 learned_addr learned my addr 172.21.15.135:0/907952818 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc276370640 1 --2- 172.21.15.135:0/907952818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015ff30 0x7fc27015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc276b71640 1 -- 172.21.15.135:0/907952818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015ff30 msgr2=0x7fc27015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc276b71640 1 --2- 172.21.15.135:0/907952818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015ff30 0x7fc27015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc276b71640 1 -- 172.21.15.135:0/907952818 >> v1:172.21.15.190:6789/0 conn(0x7fc270071ec0 legacy=0x7fc2700cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:10.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.465+0000 7fc276b71640 1 -- 172.21.15.135:0/907952818 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27006e450 con 0x7fc2700cfe50 2024-04-17T06:10:10.465 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.466+0000 7fc276b71640 1 --2- 172.21.15.135:0/907952818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2700cfe50 0x7fc270072340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc268009870 tx=0x7fc26802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119ba3783ba43e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:10.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.466+0000 7fc27536e640 1 -- 172.21.15.135:0/907952818 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268035020 con 0x7fc2700cfe50 2024-04-17T06:10:10.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.466+0000 7fc27536e640 1 -- 172.21.15.135:0/907952818 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc268031640 con 0x7fc2700cfe50 2024-04-17T06:10:10.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.466+0000 7fc27536e640 1 -- 172.21.15.135:0/907952818 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26802d040 con 0x7fc2700cfe50 2024-04-17T06:10:10.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.467+0000 7fc2785d4640 1 -- 172.21.15.135:0/907952818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2700cfe50 msgr2=0x7fc270072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.467+0000 7fc2785d4640 1 --2- 172.21.15.135:0/907952818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2700cfe50 0x7fc270072340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc268009870 tx=0x7fc26802b8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.467+0000 7fc2785d4640 1 -- 172.21.15.135:0/907952818 shutdown_connections 2024-04-17T06:10:10.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.467+0000 7fc2785d4640 1 --2- 172.21.15.135:0/907952818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc2700cfe50 0x7fc270072340 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.467+0000 7fc2785d4640 1 --2- 172.21.15.135:0/907952818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015ff30 0x7fc27015cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.467+0000 7fc2785d4640 1 -- 172.21.15.135:0/907952818 >> 172.21.15.135:0/907952818 conn(0x7fc2700d0520 msgr2=0x7fc2700d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:10.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.467+0000 7fc2785d4640 1 -- 172.21.15.135:0/907952818 shutdown_connections 2024-04-17T06:10:10.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.467+0000 7fc2785d4640 1 -- 172.21.15.135:0/907952818 wait complete. 2024-04-17T06:10:10.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.468+0000 7fc2785d4640 1 Processor -- start 2024-04-17T06:10:10.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.468+0000 7fc2785d4640 1 -- start start 2024-04-17T06:10:10.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc2785d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc270071ec0 0x7fc2701452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc2785d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cfe50 0x7fc270159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc2785d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27015a0d0 0x7fc2701479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc2785d4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc27006f0b0 con 0x7fc27015a0d0 2024-04-17T06:10:10.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc2785d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc27006ef30 con 0x7fc2700cfe50 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc2785d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc27006f230 con 0x7fc270071ec0 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276b71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27015a0d0 0x7fc2701479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276b71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27015a0d0 0x7fc2701479b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55928/0 (socket says 172.21.15.135:55928) 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276b71640 1 -- 172.21.15.135:0/2301174162 learned_addr learned my addr 172.21.15.135:0/2301174162 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276370640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc270071ec0 0x7fc2701452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc275b6f640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cfe50 0x7fc270159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276b71640 1 -- 172.21.15.135:0/2301174162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc270071ec0 msgr2=0x7fc2701452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276b71640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc270071ec0 0x7fc2701452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276b71640 1 -- 172.21.15.135:0/2301174162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cfe50 msgr2=0x7fc270159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276b71640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cfe50 0x7fc270159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.469+0000 7fc276b71640 1 -- 172.21.15.135:0/2301174162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27015a700 con 0x7fc27015a0d0 2024-04-17T06:10:10.470 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.470+0000 7fc276b71640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27015a0d0 0x7fc2701479b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc26802b930 tx=0x7fc2680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:10.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.470+0000 7fc266ffd640 1 -- 172.21.15.135:0/2301174162 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268040070 con 0x7fc27015a0d0 2024-04-17T06:10:10.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.470+0000 7fc266ffd640 1 -- 172.21.15.135:0/2301174162 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2680399f0 con 0x7fc27015a0d0 2024-04-17T06:10:10.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.470+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27015a9e0 con 0x7fc27015a0d0 2024-04-17T06:10:10.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.470+0000 7fc266ffd640 1 -- 172.21.15.135:0/2301174162 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268035040 con 0x7fc27015a0d0 2024-04-17T06:10:10.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.470+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc270145c50 con 0x7fc27015a0d0 2024-04-17T06:10:10.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.471+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc270074940 con 0x7fc27015a0d0 2024-04-17T06:10:10.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.471+0000 7fc266ffd640 1 -- 172.21.15.135:0/2301174162 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc268038020 con 0x7fc27015a0d0 2024-04-17T06:10:10.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.472+0000 7fc266ffd640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc25406c920 0x7fc25406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:10.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.472+0000 7fc266ffd640 1 -- 172.21.15.135:0/2301174162 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc2680ad830 con 0x7fc27015a0d0 2024-04-17T06:10:10.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.472+0000 7fc276370640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc25406c920 0x7fc25406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:10.472 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.472+0000 7fc276370640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc25406c920 0x7fc25406ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc26c003aa0 tx=0x7fc26c009830 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:10.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.476+0000 7fc266ffd640 1 -- 172.21.15.135:0/2301174162 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc26807b210 con 0x7fc27015a0d0 2024-04-17T06:10:10.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.663+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2700cfc30 con 0x7fc27015a0d0 2024-04-17T06:10:10.661 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.663+0000 7fc266ffd640 1 -- 172.21.15.135:0/2301174162 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fc26807f510 con 0x7fc27015a0d0 2024-04-17T06:10:10.662 INFO:teuthology.orchestra.run.smithi135.stderr:listed 0 entries 2024-04-17T06:10:10.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc25406c920 msgr2=0x7fc25406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc25406c920 0x7fc25406ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc26c003aa0 tx=0x7fc26c009830 comp rx=0 tx=0).stop 2024-04-17T06:10:10.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27015a0d0 msgr2=0x7fc2701479b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:10.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27015a0d0 0x7fc2701479b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc26802b930 tx=0x7fc2680046c0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 shutdown_connections 2024-04-17T06:10:10.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc25406c920 0x7fc25406ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.664 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc27015a0d0 0x7fc2701479b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cfe50 0x7fc270159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 --2- 172.21.15.135:0/2301174162 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc270071ec0 0x7fc2701452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:10.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 >> 172.21.15.135:0/2301174162 conn(0x7fc2700d0520 msgr2=0x7fc27006f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:10.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 shutdown_connections 2024-04-17T06:10:10.665 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:10.666+0000 7fc2785d4640 1 -- 172.21.15.135:0/2301174162 wait complete. 2024-04-17T06:10:10.675 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-17T06:10:10.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:10.676 DEBUG:teuthology.orchestra.run.smithi135:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-17T06:10:10.706 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:10:10.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:10.707 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-17T06:10:10.763 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:10.763 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:10.783 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:10.783 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:10.843 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:10.843 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:10.901 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:11.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.007+0000 7f6442500640 1 Processor -- start 2024-04-17T06:10:11.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.007+0000 7f6442500640 1 -- start start 2024-04-17T06:10:11.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f6442500640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15cac0 0x7f643c06e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f6442500640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f643c15cea0 con 0x7f643c0d03d0 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f6442500640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f643c0d0940 con 0x7f643c06d020 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f6442500640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f643c06c540 con 0x7f643c15cac0 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f6440a9d640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f643c0d03d0 0x7f643c0add30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42352/0 (socket says 172.21.15.135:42352) 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f6440a9d640 1 -- 172.21.15.135:0/3396199862 learned_addr learned my addr 172.21.15.135:0/3396199862 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643bfff640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15cac0 0x7f643c06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 440866251 0 0) 0x7f643c15cea0 con 0x7f643c0d03d0 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6424003520 con 0x7f643c0d03d0 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2547256157 0 0) 0x7f6424003520 con 0x7f643c0d03d0 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15cac0 msgr2=0x7f643c06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15cac0 0x7f643c06e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 >> v1:172.21.15.190:6789/0 conn(0x7f643c06d020 legacy=0x7f643c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c06c820 con 0x7f643c0d03d0 2024-04-17T06:10:11.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f64340019f0 con 0x7f643c0d03d0 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 >> v1:172.21.15.135:6789/0 conn(0x7f643c0d03d0 legacy=0x7f643c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64240039e0 0x7f6424003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6424004800 0x7f6424004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15cac0 0x7f64240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f643c06c540 con 0x7f64240039e0 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f642400ba70 con 0x7f6424004800 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.008+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f642400bc70 con 0x7f643c15cac0 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.009+0000 7f643bfff640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64240039e0 0x7f6424003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.009+0000 7f643b7fe640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6424004800 0x7f6424004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.009+0000 7f6440a9d640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15cac0 0x7f64240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.009+0000 7f643bfff640 1 -- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15cac0 msgr2=0x7f64240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.009+0000 7f643bfff640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c15cac0 0x7f64240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.009+0000 7f643bfff640 1 -- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6424004800 msgr2=0x7f6424004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:11.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.009+0000 7f643bfff640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6424004800 0x7f6424004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.009+0000 7f643bfff640 1 -- 172.21.15.135:0/3396199862 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6424003520 con 0x7f64240039e0 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f643bfff640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64240039e0 0x7f6424003e00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f642c009a90 tx=0x7f6424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4a5e7bcfdb31c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c039070 con 0x7f64240039e0 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c02c680 con 0x7f64240039e0 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f643affd640 1 -- 172.21.15.135:0/3396199862 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c009690 con 0x7f64240039e0 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f6442500640 1 -- 172.21.15.135:0/3396199862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64240039e0 msgr2=0x7f6424003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f6442500640 1 --2- 172.21.15.135:0/3396199862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64240039e0 0x7f6424003e00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f642c009a90 tx=0x7f6424005830 comp rx=0 tx=0).stop 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f6440a9d640 1 -- 172.21.15.135:0/3396199862 reap_dead start 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f6442500640 1 -- 172.21.15.135:0/3396199862 shutdown_connections 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f6442500640 1 -- 172.21.15.135:0/3396199862 >> 172.21.15.135:0/3396199862 conn(0x7f643c0cd7e0 msgr2=0x7f643c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f6442500640 1 -- 172.21.15.135:0/3396199862 shutdown_connections 2024-04-17T06:10:11.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.010+0000 7f6442500640 1 -- 172.21.15.135:0/3396199862 wait complete. 2024-04-17T06:10:11.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.011+0000 7f6442500640 1 Processor -- start 2024-04-17T06:10:11.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6442500640 1 -- start start 2024-04-17T06:10:11.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6442500640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64240039e0 0x7f643c14c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6442500640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c0ca1d0 0x7f643c0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6442500640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643c0b4390 0x7f643c0b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6442500640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f643c070c00 con 0x7f643c0b4390 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6442500640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f643c070a80 con 0x7f64240039e0 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6442500640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f643c070d80 con 0x7f643c0ca1d0 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6440a9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643c0b4390 0x7f643c0b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6440a9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643c0b4390 0x7f643c0b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55944/0 (socket says 172.21.15.135:55944) 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.012+0000 7f6440a9d640 1 -- 172.21.15.135:0/3599182882 learned_addr learned my addr 172.21.15.135:0/3599182882 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f643bfff640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64240039e0 0x7f643c14c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f643b7fe640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c0ca1d0 0x7f643c0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6440a9d640 1 -- 172.21.15.135:0/3599182882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c0ca1d0 msgr2=0x7f643c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6440a9d640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c0ca1d0 0x7f643c0b3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6440a9d640 1 -- 172.21.15.135:0/3599182882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64240039e0 msgr2=0x7f643c14c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6440a9d640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64240039e0 0x7f643c14c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6440a9d640 1 -- 172.21.15.135:0/3599182882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c15d500 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6440a9d640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643c0b4390 0x7f643c0b4840 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6434000f80 tx=0x7f6434007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6438ff9640 1 -- 172.21.15.135:0/3599182882 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643400abe0 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6438ff9640 1 -- 172.21.15.135:0/3599182882 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6434019070 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c15d7e0 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6438ff9640 1 -- 172.21.15.135:0/3599182882 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340133f0 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.013+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c15dda0 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.014+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c074940 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.015+0000 7f6438ff9640 1 -- 172.21.15.135:0/3599182882 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6434013590 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.015+0000 7f6438ff9640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f642806c8f0 0x7f642806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.015+0000 7f6438ff9640 1 -- 172.21.15.135:0/3599182882 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f643408c570 con 0x7f643c0b4390 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.015+0000 7f643bfff640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f642806c8f0 0x7f642806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.016 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.016+0000 7f643bfff640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f642806c8f0 0x7f642806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f642c036000 tx=0x7f642c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.017 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.020+0000 7f6438ff9640 1 -- 172.21.15.135:0/3599182882 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f643405a5a0 con 0x7f643c0b4390 2024-04-17T06:10:11.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.196+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f643c15c1e0 con 0x7f643c0b4390 2024-04-17T06:10:11.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.198+0000 7f6438ff9640 1 -- 172.21.15.135:0/3599182882 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18950 (secure 0 0 0) 0x7f6434059ea0 con 0x7f643c0b4390 2024-04-17T06:10:11.196 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:11.196 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:01.008924+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f642806c8f0 msgr2=0x7f642806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f642806c8f0 0x7f642806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f642c036000 tx=0x7f642c030000 comp rx=0 tx=0).stop 2024-04-17T06:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643c0b4390 msgr2=0x7f643c0b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643c0b4390 0x7f643c0b4840 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6434000f80 tx=0x7f6434007bd0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 shutdown_connections 2024-04-17T06:10:11.198 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f642806c8f0 0x7f642806edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f643c0b4390 0x7f643c0b4840 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f643c0ca1d0 0x7f643c0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 --2- 172.21.15.135:0/3599182882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64240039e0 0x7f643c14c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 >> 172.21.15.135:0/3599182882 conn(0x7f643c0cd7e0 msgr2=0x7f643c0affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:11.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 shutdown_connections 2024-04-17T06:10:11.199 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.200+0000 7f6442500640 1 -- 172.21.15.135:0/3599182882 wait complete. 2024-04-17T06:10:11.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:10:11.210 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:10:11.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.318+0000 7f0585174640 1 Processor -- start 2024-04-17T06:10:11.316 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.318+0000 7f0585174640 1 -- start start 2024-04-17T06:10:11.319 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f0585174640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015cdf0 0x7f05800cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f0585174640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058006e170 con 0x7f05800725d0 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f0585174640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058006e450 con 0x7f058015ff30 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f0585174640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f058006df20 con 0x7f058015cdf0 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f0577fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f05800725d0 0x7f05800cf740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42364/0 (socket says 172.21.15.135:42364) 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f0577fff640 1 -- 172.21.15.135:0/3568826709 learned_addr learned my addr 172.21.15.135:0/3568826709 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057f59e640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015cdf0 0x7f05800cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824020884 0 0) 0x7f058006e170 con 0x7f05800725d0 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f056c003520 con 0x7f05800725d0 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 787623750 0 0) 0x7f056c003520 con 0x7f05800725d0 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015cdf0 msgr2=0x7f05800cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015cdf0 0x7f05800cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.320 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 >> v1:172.21.15.190:6789/0 conn(0x7f058015ff30 legacy=0x7f0580071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058006e730 con 0x7f05800725d0 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f0568001670 con 0x7f05800725d0 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 >> v1:172.21.15.135:6789/0 conn(0x7f05800725d0 legacy=0x7f05800cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015cdf0 0x7f056c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f056c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c004d40 0x7f056c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f058006df20 con 0x7f056c003f20 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f056c00ba20 con 0x7f056c004d40 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.319+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f056c00bc20 con 0x7f058015cdf0 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.320+0000 7f0577fff640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f056c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.320+0000 7f057ed9d640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015cdf0 0x7f056c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.320+0000 7f057f59e640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c004d40 0x7f056c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.320+0000 7f0577fff640 1 -- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015cdf0 msgr2=0x7f056c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:11.321 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.320+0000 7f0577fff640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058015cdf0 0x7f056c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.320+0000 7f0577fff640 1 -- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c004d40 msgr2=0x7f056c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.320+0000 7f0577fff640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c004d40 0x7f056c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.320+0000 7f0577fff640 1 -- 172.21.15.135:0/3568826709 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c003520 con 0x7f056c003f20 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f0577fff640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f056c004800 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f056800c290 tx=0x7f056c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e559e3394b4596de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568007d50 con 0x7f056c003f20 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0568030070 con 0x7f056c003f20 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f057e59c640 1 -- 172.21.15.135:0/3568826709 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568005d70 con 0x7f056c003f20 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f0585174640 1 -- 172.21.15.135:0/3568826709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 msgr2=0x7f056c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f0585174640 1 --2- 172.21.15.135:0/3568826709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f056c004800 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f056800c290 tx=0x7f056c005830 comp rx=0 tx=0).stop 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f057f59e640 1 -- 172.21.15.135:0/3568826709 reap_dead start 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f0585174640 1 -- 172.21.15.135:0/3568826709 shutdown_connections 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f0585174640 1 -- 172.21.15.135:0/3568826709 >> 172.21.15.135:0/3568826709 conn(0x7f05800d0520 msgr2=0x7f05800d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:11.322 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f0585174640 1 -- 172.21.15.135:0/3568826709 shutdown_connections 2024-04-17T06:10:11.323 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.321+0000 7f0585174640 1 -- 172.21.15.135:0/3568826709 wait complete. 2024-04-17T06:10:11.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f0585174640 1 Processor -- start 2024-04-17T06:10:11.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f0585174640 1 -- start start 2024-04-17T06:10:11.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f0585174640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f058013ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f0585174640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058013e6e0 0x7f0580140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.325 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f0585174640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0580140a70 0x7f058015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f0585174640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05800d01e0 con 0x7f056c003f20 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f0585174640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f05800d0060 con 0x7f0580140a70 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f0585174640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f058006eaa0 con 0x7f058013e6e0 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f057ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f058013ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f057ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f058013ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55960/0 (socket says 172.21.15.135:55960) 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.323+0000 7f057ed9d640 1 -- 172.21.15.135:0/491819252 learned_addr learned my addr 172.21.15.135:0/491819252 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f0577fff640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058013e6e0 0x7f0580140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f057f59e640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0580140a70 0x7f058015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f057ed9d640 1 -- 172.21.15.135:0/491819252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058013e6e0 msgr2=0x7f0580140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f057ed9d640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058013e6e0 0x7f0580140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f057ed9d640 1 -- 172.21.15.135:0/491819252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0580140a70 msgr2=0x7f058015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f057ed9d640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0580140a70 0x7f058015ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.326 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f057ed9d640 1 -- 172.21.15.135:0/491819252 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058013f490 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f057ed9d640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f058013ee20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f057000f840 tx=0x7f057000fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f05777fe640 1 -- 172.21.15.135:0/491819252 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05700052e0 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f05777fe640 1 -- 172.21.15.135:0/491819252 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570018070 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058013f770 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f05777fe640 1 -- 172.21.15.135:0/491819252 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057000d970 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.324+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058013fd30 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.325+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0580074940 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.326+0000 7f05777fe640 1 -- 172.21.15.135:0/491819252 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f057000db10 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.326+0000 7f05777fe640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f056006c8f0 0x7f056006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.326+0000 7f05777fe640 1 -- 172.21.15.135:0/491819252 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f057008bf30 con 0x7f056c003f20 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.326+0000 7f0577fff640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f056006c8f0 0x7f056006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.327 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.326+0000 7f0577fff640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f056006c8f0 0x7f056006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0568034000 tx=0x7f0568009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.328 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.330+0000 7f05777fe640 1 -- 172.21.15.135:0/491819252 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0570059ff0 con 0x7f056c003f20 2024-04-17T06:10:11.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.506+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0580161070 con 0x7f056c003f20 2024-04-17T06:10:11.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.506+0000 7f05777fe640 1 -- 172.21.15.135:0/491819252 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f05700598f0 con 0x7f056c003f20 2024-04-17T06:10:11.504 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f056006c8f0 msgr2=0x7f056006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f056006c8f0 0x7f056006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0568034000 tx=0x7f0568009030 comp rx=0 tx=0).stop 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 msgr2=0x7f058013ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f058013ee20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f057000f840 tx=0x7f057000fd00 comp rx=0 tx=0).stop 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 shutdown_connections 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0580140a70 0x7f058015ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f058013e6e0 0x7f0580140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f056006c8f0 0x7f056006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 --2- 172.21.15.135:0/491819252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f056c003f20 0x7f058013ee20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 >> 172.21.15.135:0/491819252 conn(0x7f05800d0520 msgr2=0x7f0580160320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 shutdown_connections 2024-04-17T06:10:11.507 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.508+0000 7f0585174640 1 -- 172.21.15.135:0/491819252 wait complete. 2024-04-17T06:10:11.517 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:10:11.623 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.625+0000 7f4c74a47640 1 Processor -- start 2024-04-17T06:10:11.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.626+0000 7f4c74a47640 1 -- start start 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.626+0000 7f4c74a47640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4c700d43d0 0x7f4c700d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.626+0000 7f4c74a47640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d1810 0x7f4c700b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.626+0000 7f4c74a47640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c700d30a0 con 0x7f4c7015ec00 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.626+0000 7f4c74a47640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4c700d1ca0 con 0x7f4c700d43d0 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c74a47640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7015efe0 con 0x7f4c700d1810 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c6dd9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4c700d43d0 0x7f4c700d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c6dd9b640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4c700d43d0 0x7f4c700d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55970/0 (socket says 172.21.15.135:55970) 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c6dd9b640 1 -- 172.21.15.135:0/680527180 learned_addr learned my addr 172.21.15.135:0/680527180 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c6ed9d640 1 --2- 172.21.15.135:0/680527180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d1810 0x7f4c700b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c6dd9b640 1 -- 172.21.15.135:0/680527180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d1810 msgr2=0x7f4c700b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:11.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c6dd9b640 1 --2- 172.21.15.135:0/680527180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d1810 0x7f4c700b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c6dd9b640 1 -- 172.21.15.135:0/680527180 >> v1:172.21.15.190:6789/0 conn(0x7f4c7015ec00 legacy=0x7f4c700d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.627+0000 7f4c6dd9b640 1 -- 172.21.15.135:0/680527180 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c700d3330 con 0x7f4c700d43d0 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c6dd9b640 1 --2- 172.21.15.135:0/680527180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d43d0 0x7f4c700d12d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009870 tx=0x7f4c6402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df40e473bdacdb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c6d59a640 1 -- 172.21.15.135:0/680527180 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6402f840 con 0x7f4c700d43d0 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c6d59a640 1 -- 172.21.15.135:0/680527180 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6402f9e0 con 0x7f4c700d43d0 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c6d59a640 1 -- 172.21.15.135:0/680527180 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6402fdb0 con 0x7f4c700d43d0 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c74a47640 1 -- 172.21.15.135:0/680527180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d43d0 msgr2=0x7f4c700d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c74a47640 1 --2- 172.21.15.135:0/680527180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d43d0 0x7f4c700d12d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009870 tx=0x7f4c6402ba20 comp rx=0 tx=0).stop 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c74a47640 1 -- 172.21.15.135:0/680527180 shutdown_connections 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c74a47640 1 --2- 172.21.15.135:0/680527180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d1810 0x7f4c700b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c74a47640 1 --2- 172.21.15.135:0/680527180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d43d0 0x7f4c700d12d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.628+0000 7f4c74a47640 1 -- 172.21.15.135:0/680527180 >> 172.21.15.135:0/680527180 conn(0x7f4c7006cc90 msgr2=0x7f4c7006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:11.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.629+0000 7f4c74a47640 1 -- 172.21.15.135:0/680527180 shutdown_connections 2024-04-17T06:10:11.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.629+0000 7f4c74a47640 1 -- 172.21.15.135:0/680527180 wait complete. 2024-04-17T06:10:11.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.630+0000 7f4c74a47640 1 Processor -- start 2024-04-17T06:10:11.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.630+0000 7f4c74a47640 1 -- start start 2024-04-17T06:10:11.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c74a47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d1810 0x7f4c700cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c74a47640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d43d0 0x7f4c701f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c74a47640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7015ec00 0x7f4c700c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c74a47640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c700d38f0 con 0x7f4c700d1810 2024-04-17T06:10:11.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c74a47640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4c700d3f40 con 0x7f4c7015ec00 2024-04-17T06:10:11.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c74a47640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4c700d3bd0 con 0x7f4c700d43d0 2024-04-17T06:10:11.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d1810 0x7f4c700cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d1810 0x7f4c700cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55984/0 (socket says 172.21.15.135:55984) 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 -- 172.21.15.135:0/656978545 learned_addr learned my addr 172.21.15.135:0/656978545 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6dd9b640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d43d0 0x7f4c701f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6ed9d640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7015ec00 0x7f4c700c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 -- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d43d0 msgr2=0x7f4c701f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d43d0 0x7f4c701f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 -- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7015ec00 msgr2=0x7f4c700c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7015ec00 0x7f4c700c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 -- 172.21.15.135:0/656978545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c700ccf70 con 0x7f4c700d1810 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6dd9b640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d43d0 0x7f4c701f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6ed9d640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7015ec00 0x7f4c700c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c6e59c640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d1810 0x7f4c700cca30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c3c0 tx=0x7f4c6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c5effd640 1 -- 172.21.15.135:0/656978545 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600052d0 con 0x7f4c700d1810 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c5effd640 1 -- 172.21.15.135:0/656978545 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c60005490 con 0x7f4c700d1810 2024-04-17T06:10:11.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c700cd240 con 0x7f4c700d1810 2024-04-17T06:10:11.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c5effd640 1 -- 172.21.15.135:0/656978545 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60007260 con 0x7f4c700d1810 2024-04-17T06:10:11.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.631+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c700cd800 con 0x7f4c700d1810 2024-04-17T06:10:11.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.632+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c70074940 con 0x7f4c700d1810 2024-04-17T06:10:11.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.633+0000 7f4c5effd640 1 -- 172.21.15.135:0/656978545 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c60007400 con 0x7f4c700d1810 2024-04-17T06:10:11.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.633+0000 7f4c5effd640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4c5006c8f0 0x7f4c5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.633+0000 7f4c5effd640 1 -- 172.21.15.135:0/656978545 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f4c6008b410 con 0x7f4c700d1810 2024-04-17T06:10:11.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.633+0000 7f4c6dd9b640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4c5006c8f0 0x7f4c5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.634+0000 7f4c6dd9b640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4c5006c8f0 0x7f4c5006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4c64036000 tx=0x7f4c64008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.638+0000 7f4c5effd640 1 -- 172.21.15.135:0/656978545 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4c60059440 con 0x7f4c700d1810 2024-04-17T06:10:11.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.812+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4c700b44c0 con 0x7f4c700d1810 2024-04-17T06:10:11.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.813+0000 7f4c5effd640 1 -- 172.21.15.135:0/656978545 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f4c60058d40 con 0x7f4c700d1810 2024-04-17T06:10:11.811 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:11.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4c5006c8f0 msgr2=0x7f4c5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4c5006c8f0 0x7f4c5006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4c64036000 tx=0x7f4c64008030 comp rx=0 tx=0).stop 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d1810 msgr2=0x7f4c700cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d1810 0x7f4c700cca30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000c3c0 tx=0x7f4c6000c880 comp rx=0 tx=0).stop 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 shutdown_connections 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7015ec00 0x7f4c700c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c700d43d0 0x7f4c701f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4c5006c8f0 0x7f4c5006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 --2- 172.21.15.135:0/656978545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c700d1810 0x7f4c700cca30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 >> 172.21.15.135:0/656978545 conn(0x7f4c7006cc90 msgr2=0x7f4c700d4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 shutdown_connections 2024-04-17T06:10:11.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.815+0000 7f4c74a47640 1 -- 172.21.15.135:0/656978545 wait complete. 2024-04-17T06:10:11.824 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:10:11.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.932+0000 7fa647367640 1 Processor -- start 2024-04-17T06:10:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.933+0000 7fa647367640 1 -- start start 2024-04-17T06:10:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.933+0000 7fa647367640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64015ee30 0x7fa64009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.933+0000 7fa647367640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa64009bbe0 0x7fa64009af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.933+0000 7fa647367640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64006db80 con 0x7fa64009b470 2024-04-17T06:10:11.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.933+0000 7fa647367640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fa64006de60 con 0x7fa64009bbe0 2024-04-17T06:10:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.933+0000 7fa647367640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa6400cce30 con 0x7fa64015ee30 2024-04-17T06:10:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa645103640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64015ee30 0x7fa64009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa644902640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa64009bbe0 0x7fa64009af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa645103640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64015ee30 0x7fa64009a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:51086/0 (socket says 172.21.15.135:51086) 2024-04-17T06:10:11.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa645904640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa64009b470 0x7fa6400d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:48240/0 (socket says 172.21.15.135:48240) 2024-04-17T06:10:11.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa645103640 1 -- 172.21.15.135:0/574690424 learned_addr learned my addr 172.21.15.135:0/574690424 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:11.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa645103640 1 -- 172.21.15.135:0/574690424 >> v2:172.21.15.190:3300/0 conn(0x7fa64009bbe0 msgr2=0x7fa64009af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:11.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa645103640 1 --2- 172.21.15.135:0/574690424 >> v2:172.21.15.190:3300/0 conn(0x7fa64009bbe0 0x7fa64009af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa645103640 1 -- 172.21.15.135:0/574690424 >> v1:172.21.15.190:6789/0 conn(0x7fa64009b470 legacy=0x7fa6400d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.934+0000 7fa645103640 1 -- 172.21.15.135:0/574690424 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6400cd110 con 0x7fa64015ee30 2024-04-17T06:10:11.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.935+0000 7fa645103640 1 --2- 172.21.15.135:0/574690424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64015ee30 0x7fa64009a9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa630009870 tx=0x7fa63002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8207e51aa0871a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.935+0000 7fa637fff640 1 -- 172.21.15.135:0/574690424 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630034070 con 0x7fa64015ee30 2024-04-17T06:10:11.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.935+0000 7fa637fff640 1 -- 172.21.15.135:0/574690424 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa630030840 con 0x7fa64015ee30 2024-04-17T06:10:11.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.935+0000 7fa637fff640 1 -- 172.21.15.135:0/574690424 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630030c10 con 0x7fa64015ee30 2024-04-17T06:10:11.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.936+0000 7fa647367640 1 -- 172.21.15.135:0/574690424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64015ee30 msgr2=0x7fa64009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.936+0000 7fa647367640 1 --2- 172.21.15.135:0/574690424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64015ee30 0x7fa64009a9f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa630009870 tx=0x7fa63002b980 comp rx=0 tx=0).stop 2024-04-17T06:10:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.936+0000 7fa647367640 1 -- 172.21.15.135:0/574690424 shutdown_connections 2024-04-17T06:10:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.936+0000 7fa647367640 1 --2- 172.21.15.135:0/574690424 >> v2:172.21.15.190:3300/0 conn(0x7fa64009bbe0 0x7fa64009af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.936+0000 7fa647367640 1 --2- 172.21.15.135:0/574690424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64015ee30 0x7fa64009a9f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.936+0000 7fa647367640 1 -- 172.21.15.135:0/574690424 >> 172.21.15.135:0/574690424 conn(0x7fa6400b41a0 msgr2=0x7fa6400b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.936+0000 7fa647367640 1 -- 172.21.15.135:0/574690424 shutdown_connections 2024-04-17T06:10:11.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.936+0000 7fa647367640 1 -- 172.21.15.135:0/574690424 wait complete. 2024-04-17T06:10:11.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.937+0000 7fa647367640 1 Processor -- start 2024-04-17T06:10:11.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.937+0000 7fa647367640 1 -- start start 2024-04-17T06:10:11.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa647367640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64009b470 0x7fa640152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa647367640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64009bbe0 0x7fa640153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa647367640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa64015ee30 0x7fa6400c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa647367640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa64006d910 con 0x7fa64015ee30 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa647367640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa64006d790 con 0x7fa64009bbe0 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa647367640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa6400ce640 con 0x7fa64009b470 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa64015ee30 0x7fa6400c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa64015ee30 0x7fa6400c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55986/0 (socket says 172.21.15.135:55986) 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 -- 172.21.15.135:0/2552224957 learned_addr learned my addr 172.21.15.135:0/2552224957 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645103640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64009b470 0x7fa640152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa644902640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64009bbe0 0x7fa640153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 -- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64009b470 msgr2=0x7fa640152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64009b470 0x7fa640152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 -- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64009bbe0 msgr2=0x7fa640153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64009bbe0 0x7fa640153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:11.938 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 -- 172.21.15.135:0/2552224957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64015ca00 con 0x7fa64015ee30 2024-04-17T06:10:11.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa644902640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64009bbe0 0x7fa640153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:11.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.938+0000 7fa645904640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa64015ee30 0x7fa6400c8230 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c780 tx=0x7fa63800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.939+0000 7fa635ffb640 1 -- 172.21.15.135:0/2552224957 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638017070 con 0x7fa64015ee30 2024-04-17T06:10:11.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.939+0000 7fa635ffb640 1 -- 172.21.15.135:0/2552224957 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63800a640 con 0x7fa64015ee30 2024-04-17T06:10:11.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.939+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6400c6310 con 0x7fa64015ee30 2024-04-17T06:10:11.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.939+0000 7fa635ffb640 1 -- 172.21.15.135:0/2552224957 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63800aa30 con 0x7fa64015ee30 2024-04-17T06:10:11.939 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.939+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6400c68d0 con 0x7fa64015ee30 2024-04-17T06:10:11.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.940+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa640074940 con 0x7fa64015ee30 2024-04-17T06:10:11.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.940+0000 7fa635ffb640 1 -- 172.21.15.135:0/2552224957 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa63801a070 con 0x7fa64015ee30 2024-04-17T06:10:11.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.940+0000 7fa635ffb640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa62406c8f0 0x7fa62406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:11.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.941+0000 7fa635ffb640 1 -- 172.21.15.135:0/2552224957 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fa63808b430 con 0x7fa64015ee30 2024-04-17T06:10:11.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.941+0000 7fa645103640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa62406c8f0 0x7fa62406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:11.940 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.941+0000 7fa645103640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa62406c8f0 0x7fa62406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa630036000 tx=0x7fa63003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:11.942 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:11.945+0000 7fa635ffb640 1 -- 172.21.15.135:0/2552224957 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa638058ea0 con 0x7fa64015ee30 2024-04-17T06:10:12.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.120+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6400d2330 con 0x7fa64015ee30 2024-04-17T06:10:12.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.120+0000 7fa635ffb640 1 -- 172.21.15.135:0/2552224957 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fa63805d1a0 con 0x7fa64015ee30 2024-04-17T06:10:12.119 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa62406c8f0 msgr2=0x7fa62406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa62406c8f0 0x7fa62406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa630036000 tx=0x7fa63003b040 comp rx=0 tx=0).stop 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa64015ee30 msgr2=0x7fa6400c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa64015ee30 0x7fa6400c8230 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c780 tx=0x7fa63800cc40 comp rx=0 tx=0).stop 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 shutdown_connections 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa62406c8f0 0x7fa62406edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa64015ee30 0x7fa6400c8230 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa64009bbe0 0x7fa640153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 --2- 172.21.15.135:0/2552224957 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa64009b470 0x7fa640152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 >> 172.21.15.135:0/2552224957 conn(0x7fa6400b41a0 msgr2=0x7fa64006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:12.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 shutdown_connections 2024-04-17T06:10:12.122 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.123+0000 7fa647367640 1 -- 172.21.15.135:0/2552224957 wait complete. 2024-04-17T06:10:12.132 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:10:12.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.239+0000 7f55cd752640 1 Processor -- start 2024-04-17T06:10:12.238 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.239+0000 7f55cd752640 1 -- start start 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.240+0000 7f55cd752640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c815ff50 0x7f55c80715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.240+0000 7f55cd752640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55c8160330 0x7f55c80d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.240+0000 7f55cd752640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c80d1460 con 0x7f55c80d1050 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.240+0000 7f55cd752640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f55c809ba20 con 0x7f55c8160330 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.240+0000 7f55cd752640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f55c80cdac0 con 0x7f55c815ff50 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.240+0000 7f55c67fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55c8160330 0x7f55c80d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.240+0000 7f55c67fc640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55c8160330 0x7f55c80d0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:55990/0 (socket says 172.21.15.135:55990) 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.240+0000 7f55c67fc640 1 -- 172.21.15.135:0/829864824 learned_addr learned my addr 172.21.15.135:0/829864824 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.241+0000 7f55c6ffd640 1 --2- 172.21.15.135:0/829864824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c815ff50 0x7f55c80715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.241+0000 7f55c67fc640 1 -- 172.21.15.135:0/829864824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c815ff50 msgr2=0x7f55c80715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.241+0000 7f55c67fc640 1 --2- 172.21.15.135:0/829864824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c815ff50 0x7f55c80715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.241+0000 7f55c67fc640 1 -- 172.21.15.135:0/829864824 >> v1:172.21.15.190:6789/0 conn(0x7f55c80d1050 legacy=0x7f55c8070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:12.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.241+0000 7f55c67fc640 1 -- 172.21.15.135:0/829864824 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c80b5130 con 0x7f55c8160330 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.241+0000 7f55c67fc640 1 --2- 172.21.15.135:0/829864824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c8160330 0x7f55c80d0b10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009870 tx=0x7f55bc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49d64d5601e929 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55c5ffb640 1 -- 172.21.15.135:0/829864824 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc035020 con 0x7f55c8160330 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55c5ffb640 1 -- 172.21.15.135:0/829864824 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55bc031640 con 0x7f55c8160330 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55c5ffb640 1 -- 172.21.15.135:0/829864824 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc02d040 con 0x7f55c8160330 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55cd752640 1 -- 172.21.15.135:0/829864824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c8160330 msgr2=0x7f55c80d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55cd752640 1 --2- 172.21.15.135:0/829864824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c8160330 0x7f55c80d0b10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009870 tx=0x7f55bc02b8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55cd752640 1 -- 172.21.15.135:0/829864824 shutdown_connections 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55cd752640 1 --2- 172.21.15.135:0/829864824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c8160330 0x7f55c80d0b10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55cd752640 1 --2- 172.21.15.135:0/829864824 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c815ff50 0x7f55c80715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55cd752640 1 -- 172.21.15.135:0/829864824 >> 172.21.15.135:0/829864824 conn(0x7f55c80b4240 msgr2=0x7f55c80b4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55cd752640 1 -- 172.21.15.135:0/829864824 shutdown_connections 2024-04-17T06:10:12.241 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.242+0000 7f55cd752640 1 -- 172.21.15.135:0/829864824 wait complete. 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55cd752640 1 Processor -- start 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55cd752640 1 -- start start 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55cd752640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c80d1050 0x7f55c80b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55cd752640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c815ff50 0x7f55c80cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55cd752640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c8160330 0x7f55c81f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55cd752640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55c80cd750 con 0x7f55c815ff50 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55cd752640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55c80cd470 con 0x7f55c80d1050 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55cd752640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f55c80af460 con 0x7f55c8160330 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55c67fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c815ff50 0x7f55c80cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55c67fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c815ff50 0x7f55c80cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:56004/0 (socket says 172.21.15.135:56004) 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.244+0000 7f55c67fc640 1 -- 172.21.15.135:0/4108374356 learned_addr learned my addr 172.21.15.135:0/4108374356 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c77fe640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c8160330 0x7f55c81f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c6ffd640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c80d1050 0x7f55c80b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c67fc640 1 -- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c8160330 msgr2=0x7f55c81f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c67fc640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c8160330 0x7f55c81f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c67fc640 1 -- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c80d1050 msgr2=0x7f55c80b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c67fc640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c80d1050 0x7f55c80b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c67fc640 1 -- 172.21.15.135:0/4108374356 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c80b84d0 con 0x7f55c815ff50 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c77fe640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c8160330 0x7f55c81f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55c67fc640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c815ff50 0x7f55c80cc1d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02b930 tx=0x7f55bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55af7fe640 1 -- 172.21.15.135:0/4108374356 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc040070 con 0x7f55c815ff50 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55af7fe640 1 -- 172.21.15.135:0/4108374356 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55bc039c10 con 0x7f55c815ff50 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55af7fe640 1 -- 172.21.15.135:0/4108374356 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc035040 con 0x7f55c815ff50 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c80b5a00 con 0x7f55c815ff50 2024-04-17T06:10:12.245 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.245+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c81f26e0 con 0x7f55c815ff50 2024-04-17T06:10:12.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.246+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c81f24d0 con 0x7f55c815ff50 2024-04-17T06:10:12.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.247+0000 7f55af7fe640 1 -- 172.21.15.135:0/4108374356 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55bc04b060 con 0x7f55c815ff50 2024-04-17T06:10:12.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.247+0000 7f55af7fe640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f55b406c8f0 0x7f55b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.246 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.247+0000 7f55c6ffd640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f55b406c8f0 0x7f55b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.247+0000 7f55af7fe640 1 -- 172.21.15.135:0/4108374356 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f55bc0ada50 con 0x7f55c815ff50 2024-04-17T06:10:12.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.248+0000 7f55c6ffd640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f55b406c8f0 0x7f55b406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f55b80070b0 tx=0x7f55b8007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.252+0000 7f55af7fe640 1 -- 172.21.15.135:0/4108374356 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55c80d1460 con 0x7f55c815ff50 2024-04-17T06:10:12.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.426+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f55c80cc710 con 0x7f55c815ff50 2024-04-17T06:10:12.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.427+0000 7f55af7fe640 1 -- 172.21.15.135:0/4108374356 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f55bc02f300 con 0x7f55c815ff50 2024-04-17T06:10:12.425 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:12.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f55b406c8f0 msgr2=0x7f55b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f55b406c8f0 0x7f55b406edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f55b80070b0 tx=0x7f55b8007040 comp rx=0 tx=0).stop 2024-04-17T06:10:12.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c815ff50 msgr2=0x7f55c80cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c815ff50 0x7f55c80cc1d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f55bc02b930 tx=0x7f55bc0046c0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 shutdown_connections 2024-04-17T06:10:12.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55c8160330 0x7f55c81f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f55b406c8f0 0x7f55b406edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55c815ff50 0x7f55c80cc1d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 --2- 172.21.15.135:0/4108374356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55c80d1050 0x7f55c80b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 >> 172.21.15.135:0/4108374356 conn(0x7f55c80b4240 msgr2=0x7f55c815bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:12.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 shutdown_connections 2024-04-17T06:10:12.428 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.429+0000 7f55cd752640 1 -- 172.21.15.135:0/4108374356 wait complete. 2024-04-17T06:10:12.439 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T06:10:12.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.544+0000 7fc16999d640 1 Processor -- start 2024-04-17T06:10:12.553 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.544+0000 7fc16999d640 1 -- start start 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc16999d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16409a9f0 0x7fc16409add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc16999d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc1640c78d0 0x7fc1640cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc16999d640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc16409b310 con 0x7fc16415cc60 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc16999d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc1640d0c90 con 0x7fc1640c78d0 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc16999d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc1640d0f70 con 0x7fc16409a9f0 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc15a7fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc16415cc60 0x7fc1640cd370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42374/0 (socket says 172.21.15.135:42374) 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc15a7fc640 1 -- 172.21.15.135:0/969865291 learned_addr learned my addr 172.21.15.135:0/969865291 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc162ffd640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16409a9f0 0x7fc16409add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc1637fe640 1 --2- 172.21.15.135:0/969865291 >> v2:172.21.15.190:3300/0 conn(0x7fc1640c78d0 0x7fc1640cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.554 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.545+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568660632 0 0) 0x7fc16409b310 con 0x7fc16415cc60 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc150003520 con 0x7fc16415cc60 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 96044791 0 0) 0x7fc150003520 con 0x7fc16415cc60 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16409a9f0 msgr2=0x7fc16409add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16409a9f0 0x7fc16409add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 >> v2:172.21.15.190:3300/0 conn(0x7fc1640c78d0 msgr2=0x7fc1640cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 --2- 172.21.15.135:0/969865291 >> v2:172.21.15.190:3300/0 conn(0x7fc1640c78d0 0x7fc1640cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1640d1250 con 0x7fc16415cc60 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1637fe640 1 --2- 172.21.15.135:0/969865291 >> v2:172.21.15.190:3300/0 conn(0x7fc1640c78d0 0x7fc1640cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fc14c001670 con 0x7fc16415cc60 2024-04-17T06:10:12.555 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 >> v1:172.21.15.135:6789/0 conn(0x7fc16415cc60 legacy=0x7fc1640cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16409a9f0 0x7fc1500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc150003f30 0x7fc150004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc150004d40 0x7fc1500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1640d0c90 con 0x7fc150004d40 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1640d0f70 con 0x7fc150003f30 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc15000ba10 con 0x7fc16409a9f0 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1637fe640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc150004d40 0x7fc1500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc162ffd640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16409a9f0 0x7fc1500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.556 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc15a7fc640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc150003f30 0x7fc150004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1637fe640 1 -- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16409a9f0 msgr2=0x7fc1500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1637fe640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16409a9f0 0x7fc1500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1637fe640 1 -- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc150003f30 msgr2=0x7fc150004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1637fe640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc150003f30 0x7fc150004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.546+0000 7fc1637fe640 1 -- 172.21.15.135:0/969865291 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc150003520 con 0x7fc150004d40 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.547+0000 7fc1637fe640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc150004d40 0x7fc1500051c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc1640c7d60 tx=0x7fc150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665f7196eeb8cbe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.547+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c034070 con 0x7fc150004d40 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.547+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15c030680 con 0x7fc150004d40 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.547+0000 7fc1627fc640 1 -- 172.21.15.135:0/969865291 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00f040 con 0x7fc150004d40 2024-04-17T06:10:12.557 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.548+0000 7fc16999d640 1 -- 172.21.15.135:0/969865291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc150004d40 msgr2=0x7fc1500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.548+0000 7fc16999d640 1 --2- 172.21.15.135:0/969865291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc150004d40 0x7fc1500051c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc1640c7d60 tx=0x7fc150005830 comp rx=0 tx=0).stop 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.548+0000 7fc1637fe640 1 -- 172.21.15.135:0/969865291 reap_dead start 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.548+0000 7fc16999d640 1 -- 172.21.15.135:0/969865291 shutdown_connections 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.548+0000 7fc16999d640 1 -- 172.21.15.135:0/969865291 >> 172.21.15.135:0/969865291 conn(0x7fc1640b41a0 msgr2=0x7fc1640b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.548+0000 7fc16999d640 1 -- 172.21.15.135:0/969865291 shutdown_connections 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.548+0000 7fc16999d640 1 -- 172.21.15.135:0/969865291 wait complete. 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.549+0000 7fc16999d640 1 Processor -- start 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.549+0000 7fc16999d640 1 -- start start 2024-04-17T06:10:12.558 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc16999d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc150004d40 0x7fc1640cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc16999d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1640ca350 0x7fc164071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc16999d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640b6050 0x7fc1640b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc16999d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc1640cca60 con 0x7fc1640b6050 2024-04-17T06:10:12.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc16999d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc1640c63a0 con 0x7fc1640ca350 2024-04-17T06:10:12.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc16999d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc16409bdf0 con 0x7fc150004d40 2024-04-17T06:10:12.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640b6050 0x7fc1640b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640b6050 0x7fc1640b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:56012/0 (socket says 172.21.15.135:56012) 2024-04-17T06:10:12.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 -- 172.21.15.135:0/816422001 learned_addr learned my addr 172.21.15.135:0/816422001 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:12.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc162ffd640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc150004d40 0x7fc1640cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc15a7fc640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1640ca350 0x7fc164071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 -- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc150004d40 msgr2=0x7fc1640cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc150004d40 0x7fc1640cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 -- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1640ca350 msgr2=0x7fc164071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1640ca350 0x7fc164071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.560 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 -- 172.21.15.135:0/816422001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1640cb0f0 con 0x7fc1640b6050 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc15a7fc640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1640ca350 0x7fc164071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc1637fe640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640b6050 0x7fc1640b64b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02bfe0 tx=0x7fc15c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc160ff9640 1 -- 172.21.15.135:0/816422001 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c049070 con 0x7fc1640b6050 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc160ff9640 1 -- 172.21.15.135:0/816422001 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15c00f000 con 0x7fc1640b6050 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.550+0000 7fc160ff9640 1 -- 172.21.15.135:0/816422001 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c034040 con 0x7fc1640b6050 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.551+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1640cb3d0 con 0x7fc1640b6050 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.551+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc164158ad0 con 0x7fc1640b6050 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.551+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc164074940 con 0x7fc1640b6050 2024-04-17T06:10:12.561 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.552+0000 7fc160ff9640 1 -- 172.21.15.135:0/816422001 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc15c037070 con 0x7fc1640b6050 2024-04-17T06:10:12.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.552+0000 7fc160ff9640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc14406c8f0 0x7fc14406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.552+0000 7fc160ff9640 1 -- 172.21.15.135:0/816422001 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc15c0ad590 con 0x7fc1640b6050 2024-04-17T06:10:12.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.552+0000 7fc162ffd640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc14406c8f0 0x7fc14406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.553+0000 7fc162ffd640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc14406c8f0 0x7fc14406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc154009fd0 tx=0x7fc15400b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.557+0000 7fc160ff9640 1 -- 172.21.15.135:0/816422001 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc15c07b530 con 0x7fc1640b6050 2024-04-17T06:10:12.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.745+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc1640c80c0 con 0x7fc1640b6050 2024-04-17T06:10:12.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.745+0000 7fc160ff9640 1 -- 172.21.15.135:0/816422001 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fc15c07ae30 con 0x7fc1640b6050 2024-04-17T06:10:12.743 INFO:teuthology.orchestra.run.smithi135.stderr:listed 0 entries 2024-04-17T06:10:12.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc14406c8f0 msgr2=0x7fc14406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc14406c8f0 0x7fc14406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc154009fd0 tx=0x7fc15400b040 comp rx=0 tx=0).stop 2024-04-17T06:10:12.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640b6050 msgr2=0x7fc1640b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640b6050 0x7fc1640b64b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc15c02bfe0 tx=0x7fc15c004730 comp rx=0 tx=0).stop 2024-04-17T06:10:12.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 shutdown_connections 2024-04-17T06:10:12.746 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc14406c8f0 0x7fc14406edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc1640b6050 0x7fc1640b64b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1640ca350 0x7fc164071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 --2- 172.21.15.135:0/816422001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc150004d40 0x7fc1640cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 >> 172.21.15.135:0/816422001 conn(0x7fc1640b41a0 msgr2=0x7fc1640c5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:12.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 shutdown_connections 2024-04-17T06:10:12.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.747+0000 7fc16999d640 1 -- 172.21.15.135:0/816422001 wait complete. 2024-04-17T06:10:12.756 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-17T06:10:12.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:12.757 DEBUG:teuthology.orchestra.run.smithi190:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-17T06:10:12.793 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2024-04-17T06:10:12.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:12.793 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-17T06:10:12.851 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-17T06:10:12.851 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:10:12.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.958+0000 7f9bb5235640 1 Processor -- start 2024-04-17T06:10:12.956 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.958+0000 7f9bb5235640 1 -- start start 2024-04-17T06:10:12.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.958+0000 7f9bb5235640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9bb009a9f0 0x7f9bb009add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.958+0000 7f9bb5235640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb015cc60 0x7f9bb00cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.958+0000 7f9bb5235640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb009b310 con 0x7f9bb00d0490 2024-04-17T06:10:12.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.958+0000 7f9bb5235640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9bb00d0c90 con 0x7f9bb009a9f0 2024-04-17T06:10:12.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.958+0000 7f9bb5235640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb00d0f70 con 0x7f9bb015cc60 2024-04-17T06:10:12.957 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9baed9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9bb009a9f0 0x7f9bb009add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9bae59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb015cc60 0x7f9bb00cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9baf59e640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9bb00d0490 0x7f9bb00cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:48252/0 (socket says 172.21.15.135:48252) 2024-04-17T06:10:12.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9baf59e640 1 -- 172.21.15.135:0/2829553318 learned_addr learned my addr 172.21.15.135:0/2829553318 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:12.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9baed9d640 1 -- 172.21.15.135:0/2829553318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb015cc60 msgr2=0x7f9bb00cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:12.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9baed9d640 1 --2- 172.21.15.135:0/2829553318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb015cc60 0x7f9bb00cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9baed9d640 1 -- 172.21.15.135:0/2829553318 >> v1:172.21.15.190:6789/0 conn(0x7f9bb00d0490 legacy=0x7f9bb00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9baed9d640 1 -- 172.21.15.135:0/2829553318 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb00d1250 con 0x7f9bb009a9f0 2024-04-17T06:10:12.958 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.959+0000 7f9bae59c640 1 --2- 172.21.15.135:0/2829553318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb015cc60 0x7f9bb00cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.960+0000 7f9baed9d640 1 --2- 172.21.15.135:0/2829553318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 0x7f9bb009add0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80097a0 tx=0x7f9ba802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8606adada8d051ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.960+0000 7f9badd9b640 1 -- 172.21.15.135:0/2829553318 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8035020 con 0x7f9bb009a9f0 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.960+0000 7f9badd9b640 1 -- 172.21.15.135:0/2829553318 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba8031810 con 0x7f9bb009a9f0 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.960+0000 7f9badd9b640 1 -- 172.21.15.135:0/2829553318 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8031be0 con 0x7f9bb009a9f0 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.961+0000 7f9bb5235640 1 -- 172.21.15.135:0/2829553318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 msgr2=0x7f9bb009add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.961+0000 7f9bb5235640 1 --2- 172.21.15.135:0/2829553318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 0x7f9bb009add0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80097a0 tx=0x7f9ba802b8b0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.961+0000 7f9bb5235640 1 -- 172.21.15.135:0/2829553318 shutdown_connections 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.961+0000 7f9bb5235640 1 --2- 172.21.15.135:0/2829553318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb015cc60 0x7f9bb00cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.961+0000 7f9bb5235640 1 --2- 172.21.15.135:0/2829553318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 0x7f9bb009add0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.961+0000 7f9bb5235640 1 -- 172.21.15.135:0/2829553318 >> 172.21.15.135:0/2829553318 conn(0x7f9bb00b41a0 msgr2=0x7f9bb00b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.961+0000 7f9bb5235640 1 -- 172.21.15.135:0/2829553318 shutdown_connections 2024-04-17T06:10:12.961 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.961+0000 7f9bb5235640 1 -- 172.21.15.135:0/2829553318 wait complete. 2024-04-17T06:10:12.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.962+0000 7f9bb5235640 1 Processor -- start 2024-04-17T06:10:12.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.962+0000 7f9bb5235640 1 -- start start 2024-04-17T06:10:12.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9bb5235640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 0x7f9bb00b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9bb5235640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb00d0490 0x7f9bb00b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9bb5235640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bb015cc60 0x7f9bb00b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9bb5235640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00cca60 con 0x7f9bb015cc60 2024-04-17T06:10:12.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9bb5235640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00c63a0 con 0x7f9bb009a9f0 2024-04-17T06:10:12.964 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9bb5235640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb009bdf0 con 0x7f9bb00d0490 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9baf59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bb015cc60 0x7f9bb00b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9baf59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bb015cc60 0x7f9bb00b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:56020/0 (socket says 172.21.15.135:56020) 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9baf59e640 1 -- 172.21.15.135:0/4159548710 learned_addr learned my addr 172.21.15.135:0/4159548710 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9bae59c640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb00d0490 0x7f9bb00b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.963+0000 7f9baed9d640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 0x7f9bb00b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9baf59e640 1 -- 172.21.15.135:0/4159548710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb00d0490 msgr2=0x7f9bb00b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9baf59e640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb00d0490 0x7f9bb00b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9baf59e640 1 -- 172.21.15.135:0/4159548710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 msgr2=0x7f9bb00b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9baf59e640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 0x7f9bb00b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9baf59e640 1 -- 172.21.15.135:0/4159548710 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb00c9990 con 0x7f9bb015cc60 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9baf59e640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bb015cc60 0x7f9bb00b7850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000c340 tx=0x7f9ba000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9b977fe640 1 -- 172.21.15.135:0/4159548710 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0017070 con 0x7f9bb015cc60 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9b977fe640 1 -- 172.21.15.135:0/4159548710 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba0005ce0 con 0x7f9bb015cc60 2024-04-17T06:10:12.965 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9b977fe640 1 -- 172.21.15.135:0/4159548710 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00072d0 con 0x7f9bb015cc60 2024-04-17T06:10:12.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb00c9c70 con 0x7f9bb015cc60 2024-04-17T06:10:12.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.964+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb00ca230 con 0x7f9bb015cc60 2024-04-17T06:10:12.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.965+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb0074940 con 0x7f9bb015cc60 2024-04-17T06:10:12.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.965+0000 7f9b977fe640 1 -- 172.21.15.135:0/4159548710 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ba001a070 con 0x7f9bb015cc60 2024-04-17T06:10:12.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.966+0000 7f9b977fe640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9b9c06c8f0 0x7f9b9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:12.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.966+0000 7f9b977fe640 1 -- 172.21.15.135:0/4159548710 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9ba008b140 con 0x7f9bb015cc60 2024-04-17T06:10:12.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.966+0000 7f9baed9d640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9b9c06c8f0 0x7f9b9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:12.966 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.966+0000 7f9baed9d640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9b9c06c8f0 0x7f9b9c06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8036000 tx=0x7f9ba803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:12.968 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:12.970+0000 7f9b977fe640 1 -- 172.21.15.135:0/4159548710 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ba0058b20 con 0x7f9bb015cc60 2024-04-17T06:10:13.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.185+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bb00b3ad0 con 0x7f9bb015cc60 2024-04-17T06:10:13.183 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.185+0000 7f9b977fe640 1 -- 172.21.15.135:0/4159548710 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f9ba005ce20 con 0x7f9bb015cc60 2024-04-17T06:10:13.183 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:13.184 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:36.038772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:10:13.184 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 9 2024-04-17T06:10:13.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9b9c06c8f0 msgr2=0x7f9b9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9b9c06c8f0 0x7f9b9c06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8036000 tx=0x7f9ba803c040 comp rx=0 tx=0).stop 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bb015cc60 msgr2=0x7f9bb00b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bb015cc60 0x7f9bb00b7850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000c340 tx=0x7f9ba000c800 comp rx=0 tx=0).stop 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 shutdown_connections 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9b9c06c8f0 0x7f9b9c06edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bb015cc60 0x7f9bb00b7850 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9bb00d0490 0x7f9bb00b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 --2- 172.21.15.135:0/4159548710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9bb009a9f0 0x7f9bb00b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 >> 172.21.15.135:0/4159548710 conn(0x7f9bb00b41a0 msgr2=0x7f9bb00c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 shutdown_connections 2024-04-17T06:10:13.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.187+0000 7f9bb5235640 1 -- 172.21.15.135:0/4159548710 wait complete. 2024-04-17T06:10:13.197 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:13.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.302+0000 7f70d42b2640 1 Processor -- start 2024-04-17T06:10:13.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.302+0000 7f70d42b2640 1 -- start start 2024-04-17T06:10:13.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d42b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc15bb00 0x7f70cc0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.301 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d42b2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70cc0afdd0 0x7f70cc0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d42b2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70cc0b3a50 con 0x7f70cc0b3590 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d42b2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f70cc0b4660 con 0x7f70cc0afdd0 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d42b2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f70cc0b3d30 con 0x7f70cc15bb00 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d204e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc15bb00 0x7f70cc0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d284f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70cc0afdd0 0x7f70cc0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d204e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc15bb00 0x7f70cc0b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:51156/0 (socket says 172.21.15.135:51156) 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d204e640 1 -- 172.21.15.135:0/906844467 learned_addr learned my addr 172.21.15.135:0/906844467 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.303+0000 7f70d184d640 1 --1- 172.21.15.135:0/906844467 >> v1:172.21.15.190:6789/0 conn(0x7f70cc0b3590 0x7f70cc0af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:48262/0 (socket says 172.21.15.135:48262) 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.304+0000 7f70d204e640 1 -- 172.21.15.135:0/906844467 >> v2:172.21.15.190:3300/0 conn(0x7f70cc0afdd0 msgr2=0x7f70cc0ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.304+0000 7f70d204e640 1 --2- 172.21.15.135:0/906844467 >> v2:172.21.15.190:3300/0 conn(0x7f70cc0afdd0 0x7f70cc0ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.304+0000 7f70d204e640 1 -- 172.21.15.135:0/906844467 >> v1:172.21.15.190:6789/0 conn(0x7f70cc0b3590 legacy=0x7f70cc0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.304+0000 7f70d204e640 1 -- 172.21.15.135:0/906844467 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc0b4010 con 0x7f70cc15bb00 2024-04-17T06:10:13.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.304+0000 7f70d284f640 1 --2- 172.21.15.135:0/906844467 >> v2:172.21.15.190:3300/0 conn(0x7f70cc0afdd0 0x7f70cc0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.304+0000 7f70d204e640 1 --2- 172.21.15.135:0/906844467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc15bb00 0x7f70cc0b5150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009870 tx=0x7f70c802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d1c6fe3577d253 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d104c640 1 -- 172.21.15.135:0/906844467 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8034070 con 0x7f70cc15bb00 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d104c640 1 -- 172.21.15.135:0/906844467 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c8030840 con 0x7f70cc15bb00 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d104c640 1 -- 172.21.15.135:0/906844467 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8030c10 con 0x7f70cc15bb00 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d42b2640 1 -- 172.21.15.135:0/906844467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc15bb00 msgr2=0x7f70cc0b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d42b2640 1 --2- 172.21.15.135:0/906844467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc15bb00 0x7f70cc0b5150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009870 tx=0x7f70c802b980 comp rx=0 tx=0).stop 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d42b2640 1 -- 172.21.15.135:0/906844467 shutdown_connections 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d42b2640 1 --2- 172.21.15.135:0/906844467 >> v2:172.21.15.190:3300/0 conn(0x7f70cc0afdd0 0x7f70cc0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d42b2640 1 --2- 172.21.15.135:0/906844467 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc15bb00 0x7f70cc0b5150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.304 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d42b2640 1 -- 172.21.15.135:0/906844467 >> 172.21.15.135:0/906844467 conn(0x7f70cc0d0520 msgr2=0x7f70cc0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:13.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d42b2640 1 -- 172.21.15.135:0/906844467 shutdown_connections 2024-04-17T06:10:13.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.305+0000 7f70d42b2640 1 -- 172.21.15.135:0/906844467 wait complete. 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.306+0000 7f70d42b2640 1 Processor -- start 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.306+0000 7f70d42b2640 1 -- start start 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d42b2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70cc0afdd0 0x7f70cc1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d42b2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc0b3590 0x7f70cc0c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d42b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70cc15bb00 0x7f70cc06b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d42b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc0b5810 con 0x7f70cc15bb00 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d42b2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc0b5690 con 0x7f70cc0afdd0 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d42b2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f70cc0cec60 con 0x7f70cc0b3590 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d284f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70cc15bb00 0x7f70cc06b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.307 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d284f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70cc15bb00 0x7f70cc06b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:56022/0 (socket says 172.21.15.135:56022) 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d284f640 1 -- 172.21.15.135:0/2340421465 learned_addr learned my addr 172.21.15.135:0/2340421465 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d184d640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc0b3590 0x7f70cc0c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.307+0000 7f70d204e640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70cc0afdd0 0x7f70cc1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70d284f640 1 -- 172.21.15.135:0/2340421465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc0b3590 msgr2=0x7f70cc0c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70d284f640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc0b3590 0x7f70cc0c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70d284f640 1 -- 172.21.15.135:0/2340421465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70cc0afdd0 msgr2=0x7f70cc1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70d284f640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70cc0afdd0 0x7f70cc1561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70d284f640 1 -- 172.21.15.135:0/2340421465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc15a930 con 0x7f70cc15bb00 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70d284f640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70cc15bb00 0x7f70cc06b990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009eb0 tx=0x7f70c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70c2ffd640 1 -- 172.21.15.135:0/2340421465 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4018020 con 0x7f70cc15bb00 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70c2ffd640 1 -- 172.21.15.135:0/2340421465 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c4014480 con 0x7f70cc15bb00 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70cc15ac10 con 0x7f70cc15bb00 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70c2ffd640 1 -- 172.21.15.135:0/2340421465 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4014850 con 0x7f70cc15bb00 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.308+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70cc15b1d0 con 0x7f70cc15bb00 2024-04-17T06:10:13.308 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.309+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70cc074940 con 0x7f70cc15bb00 2024-04-17T06:10:13.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.309+0000 7f70c2ffd640 1 -- 172.21.15.135:0/2340421465 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70c4005d10 con 0x7f70cc15bb00 2024-04-17T06:10:13.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.310+0000 7f70c2ffd640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f70b006c920 0x7f70b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.309 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.310+0000 7f70c2ffd640 1 -- 172.21.15.135:0/2340421465 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f70c408cbc0 con 0x7f70cc15bb00 2024-04-17T06:10:13.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.310+0000 7f70d204e640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f70b006c920 0x7f70b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.310+0000 7f70d204e640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f70b006c920 0x7f70b006ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f70c8036000 tx=0x7f70c803c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.312 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.314+0000 7f70c2ffd640 1 -- 172.21.15.135:0/2340421465 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70c4022030 con 0x7f70cc15bb00 2024-04-17T06:10:13.487 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.489+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70cc0cde20 con 0x7f70cc15bb00 2024-04-17T06:10:13.489 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.491+0000 7f70c2ffd640 1 -- 172.21.15.135:0/2340421465 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18950 (secure 0 0 0) 0x7f70c4010040 con 0x7f70cc15bb00 2024-04-17T06:10:13.489 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:13.489 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:01.008924+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f70b006c920 msgr2=0x7f70b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.491 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f70b006c920 0x7f70b006ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f70c8036000 tx=0x7f70c803c040 comp rx=0 tx=0).stop 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70cc15bb00 msgr2=0x7f70cc06b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70cc15bb00 0x7f70cc06b990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009eb0 tx=0x7f70c400c490 comp rx=0 tx=0).stop 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 shutdown_connections 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f70b006c920 0x7f70b006ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70cc15bb00 0x7f70cc06b990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70cc0b3590 0x7f70cc0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 --2- 172.21.15.135:0/2340421465 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70cc0afdd0 0x7f70cc1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 >> 172.21.15.135:0/2340421465 conn(0x7f70cc0d0520 msgr2=0x7f70cc157c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 shutdown_connections 2024-04-17T06:10:13.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.493+0000 7f70d42b2640 1 -- 172.21.15.135:0/2340421465 wait complete. 2024-04-17T06:10:13.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:09:18.032781+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:09:18.923217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-17T06:09:22.648961+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-17T06:10:13.503 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:10:13.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.611+0000 7fd2cede9640 1 Processor -- start 2024-04-17T06:10:13.609 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.611+0000 7fd2cede9640 1 -- start start 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cede9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd2c809bbe0 0x7fd2c809b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cede9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd2c80cd630 0x7fd2c80cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cede9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c806db80 con 0x7fd2c815ee30 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cede9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c806de60 con 0x7fd2c80cd630 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cede9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c80cce30 con 0x7fd2c809bbe0 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cd386640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd2c80cd630 0x7fd2c80cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2ccb85640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fd2c815ee30 0x7fd2c809a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42390/0 (socket says 172.21.15.135:42390) 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2ccb85640 1 -- 172.21.15.135:0/439855764 learned_addr learned my addr 172.21.15.135:0/439855764 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2bffff640 1 --2- 172.21.15.135:0/439855764 >> v2:172.21.15.190:3300/0 conn(0x7fd2c809bbe0 0x7fd2c809b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cd386640 1 -- 172.21.15.135:0/439855764 >> v2:172.21.15.190:3300/0 conn(0x7fd2c809bbe0 msgr2=0x7fd2c809b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cd386640 1 --2- 172.21.15.135:0/439855764 >> v2:172.21.15.190:3300/0 conn(0x7fd2c809bbe0 0x7fd2c809b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.612 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cd386640 1 -- 172.21.15.135:0/439855764 >> v1:172.21.15.135:6789/0 conn(0x7fd2c815ee30 legacy=0x7fd2c809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2bf7fe640 1 -- 172.21.15.135:0/439855764 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361173264 0 0) 0x7fd2c806db80 con 0x7fd2c815ee30 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.612+0000 7fd2cd386640 1 -- 172.21.15.135:0/439855764 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c80cd110 con 0x7fd2c80cd630 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.613+0000 7fd2cd386640 1 --2- 172.21.15.135:0/439855764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c80cd630 0x7fd2c80cda60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8073940 tx=0x7fd2c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c80b5d8ba5df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.613+0000 7fd2bf7fe640 1 -- 172.21.15.135:0/439855764 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0039070 con 0x7fd2c80cd630 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.613+0000 7fd2bf7fe640 1 -- 172.21.15.135:0/439855764 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c002cc80 con 0x7fd2c80cd630 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.613+0000 7fd2bf7fe640 1 -- 172.21.15.135:0/439855764 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0034660 con 0x7fd2c80cd630 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.613+0000 7fd2cede9640 1 -- 172.21.15.135:0/439855764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c80cd630 msgr2=0x7fd2c80cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.613+0000 7fd2cede9640 1 --2- 172.21.15.135:0/439855764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c80cd630 0x7fd2c80cda60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8073940 tx=0x7fd2c002bed0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.614+0000 7fd2cede9640 1 -- 172.21.15.135:0/439855764 shutdown_connections 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.614+0000 7fd2cede9640 1 --2- 172.21.15.135:0/439855764 >> v2:172.21.15.190:3300/0 conn(0x7fd2c809bbe0 0x7fd2c809b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.614+0000 7fd2cede9640 1 --2- 172.21.15.135:0/439855764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c80cd630 0x7fd2c80cda60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.614+0000 7fd2cede9640 1 -- 172.21.15.135:0/439855764 >> 172.21.15.135:0/439855764 conn(0x7fd2c80b41a0 msgr2=0x7fd2c80b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.614+0000 7fd2cede9640 1 -- 172.21.15.135:0/439855764 shutdown_connections 2024-04-17T06:10:13.613 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.614+0000 7fd2cede9640 1 -- 172.21.15.135:0/439855764 wait complete. 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.615+0000 7fd2cede9640 1 Processor -- start 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.615+0000 7fd2cede9640 1 -- start start 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cede9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c809bbe0 0x7fd2c8156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cede9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2c80cd630 0x7fd2c80c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cede9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2c815ee30 0x7fd2c81f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cede9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c806d910 con 0x7fd2c809bbe0 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cede9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c806d790 con 0x7fd2c80cd630 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cede9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c809b640 con 0x7fd2c815ee30 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c809bbe0 0x7fd2c8156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c809bbe0 0x7fd2c8156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:56034/0 (socket says 172.21.15.135:56034) 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 -- 172.21.15.135:0/3694765543 learned_addr learned my addr 172.21.15.135:0/3694765543 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:13.616 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2bffff640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2c80cd630 0x7fd2c80c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cd386640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2c815ee30 0x7fd2c81f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 -- 172.21.15.135:0/3694765543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2c815ee30 msgr2=0x7fd2c81f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2c815ee30 0x7fd2c81f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 -- 172.21.15.135:0/3694765543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2c80cd630 msgr2=0x7fd2c80c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2c80cd630 0x7fd2c80c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 -- 172.21.15.135:0/3694765543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c80c9cc0 con 0x7fd2c809bbe0 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2ccb85640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c809bbe0 0x7fd2c8156530 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400c430 tx=0x7fd2c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2bd7fa640 1 -- 172.21.15.135:0/3694765543 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40074e0 con 0x7fd2c809bbe0 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2bd7fa640 1 -- 172.21.15.135:0/3694765543 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c40047e0 con 0x7fd2c809bbe0 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2bd7fa640 1 -- 172.21.15.135:0/3694765543 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4004bd0 con 0x7fd2c809bbe0 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c81f5d50 con 0x7fd2c809bbe0 2024-04-17T06:10:13.617 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.616+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c81f6310 con 0x7fd2c809bbe0 2024-04-17T06:10:13.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.617+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c8074940 con 0x7fd2c809bbe0 2024-04-17T06:10:13.618 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.618+0000 7fd2bd7fa640 1 -- 172.21.15.135:0/3694765543 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2c4011070 con 0x7fd2c809bbe0 2024-04-17T06:10:13.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.618+0000 7fd2bd7fa640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.618+0000 7fd2bd7fa640 1 -- 172.21.15.135:0/3694765543 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fd2c408ba00 con 0x7fd2c809bbe0 2024-04-17T06:10:13.619 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.618+0000 7fd2bffff640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.619+0000 7fd2bffff640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd2b406c8f0 0x7fd2b406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd2c81f8cf0 tx=0x7fd2b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.620 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.623+0000 7fd2bd7fa640 1 -- 172.21.15.135:0/3694765543 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2c4059470 con 0x7fd2c809bbe0 2024-04-17T06:10:13.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.798+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2c80d0f50 con 0x7fd2c809bbe0 2024-04-17T06:10:13.796 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.798+0000 7fd2bd7fa640 1 -- 172.21.15.135:0/3694765543 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fd2c405d770 con 0x7fd2c809bbe0 2024-04-17T06:10:13.796 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:10:13.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd2b406c8f0 msgr2=0x7fd2b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.798 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd2b406c8f0 0x7fd2b406edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd2c81f8cf0 tx=0x7fd2b8008040 comp rx=0 tx=0).stop 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c809bbe0 msgr2=0x7fd2c8156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c809bbe0 0x7fd2c8156530 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400c430 tx=0x7fd2c400c8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 shutdown_connections 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd2c815ee30 0x7fd2c81f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2c80cd630 0x7fd2c80c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 --2- 172.21.15.135:0/3694765543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2c809bbe0 0x7fd2c8156530 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 >> 172.21.15.135:0/3694765543 conn(0x7fd2c80b41a0 msgr2=0x7fd2c809a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 shutdown_connections 2024-04-17T06:10:13.799 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.800+0000 7fd2cede9640 1 -- 172.21.15.135:0/3694765543 wait complete. 2024-04-17T06:10:13.809 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.917+0000 7ff65d60d640 1 Processor -- start 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.918+0000 7ff65d60d640 1 -- start start 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.918+0000 7ff65d60d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6581601f0 0x7ff6580d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.918+0000 7ff65d60d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff65809b100 0x7ff6580ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.918+0000 7ff65d60d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6580cd790 con 0x7ff6580cd2b0 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.918+0000 7ff65d60d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff6580cda70 con 0x7ff65809b100 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.918+0000 7ff65d60d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff6580cdd50 con 0x7ff6581601f0 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.918+0000 7ff6577fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff65809b100 0x7ff6580ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff6577fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff65809b100 0x7ff6580ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:56044/0 (socket says 172.21.15.135:56044) 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff6577fe640 1 -- 172.21.15.135:0/2987295489 learned_addr learned my addr 172.21.15.135:0/2987295489 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff656ffd640 1 --2- 172.21.15.135:0/2987295489 >> v2:172.21.15.190:3300/0 conn(0x7ff6581601f0 0x7ff6580d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff6577fe640 1 -- 172.21.15.135:0/2987295489 >> v2:172.21.15.190:3300/0 conn(0x7ff6581601f0 msgr2=0x7ff6580d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff6577fe640 1 --2- 172.21.15.135:0/2987295489 >> v2:172.21.15.190:3300/0 conn(0x7ff6581601f0 0x7ff6580d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff6577fe640 1 -- 172.21.15.135:0/2987295489 >> v1:172.21.15.190:6789/0 conn(0x7ff6580cd2b0 legacy=0x7ff65809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff6577fe640 1 -- 172.21.15.135:0/2987295489 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6580ce030 con 0x7ff65809b100 2024-04-17T06:10:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff656ffd640 1 --2- 172.21.15.135:0/2987295489 >> v2:172.21.15.190:3300/0 conn(0x7ff6581601f0 0x7ff6580d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.919+0000 7ff6577fe640 1 --2- 172.21.15.135:0/2987295489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff65809b100 0x7ff6580ce830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff65806ee40 tx=0x7ff65002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b18e0368ff8594 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff6567fc640 1 -- 172.21.15.135:0/2987295489 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65002cb20 con 0x7ff65809b100 2024-04-17T06:10:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff6567fc640 1 -- 172.21.15.135:0/2987295489 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65002ccc0 con 0x7ff65809b100 2024-04-17T06:10:13.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff6567fc640 1 -- 172.21.15.135:0/2987295489 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650034610 con 0x7ff65809b100 2024-04-17T06:10:13.921 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff65d60d640 1 -- 172.21.15.135:0/2987295489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff65809b100 msgr2=0x7ff6580ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff65d60d640 1 --2- 172.21.15.135:0/2987295489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff65809b100 0x7ff6580ce830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff65806ee40 tx=0x7ff65002bf80 comp rx=0 tx=0).stop 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff65d60d640 1 -- 172.21.15.135:0/2987295489 shutdown_connections 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff65d60d640 1 --2- 172.21.15.135:0/2987295489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff65809b100 0x7ff6580ce830 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff65d60d640 1 --2- 172.21.15.135:0/2987295489 >> v2:172.21.15.190:3300/0 conn(0x7ff6581601f0 0x7ff6580d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff65d60d640 1 -- 172.21.15.135:0/2987295489 >> 172.21.15.135:0/2987295489 conn(0x7ff6580b4190 msgr2=0x7ff6580b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff65d60d640 1 -- 172.21.15.135:0/2987295489 shutdown_connections 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.920+0000 7ff65d60d640 1 -- 172.21.15.135:0/2987295489 wait complete. 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.921+0000 7ff65d60d640 1 Processor -- start 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.922+0000 7ff65d60d640 1 -- start start 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.922+0000 7ff65d60d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff65809b100 0x7ff6580b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.922+0000 7ff65d60d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6580cd2b0 0x7ff6580cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.922+0000 7ff65d60d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6581601f0 0x7ff6580cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.922+0000 7ff65d60d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6580d0480 con 0x7ff6581601f0 2024-04-17T06:10:13.922 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.922+0000 7ff65d60d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff6580d0300 con 0x7ff6580cd2b0 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.922+0000 7ff65d60d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff6580b1aa0 con 0x7ff65809b100 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6581601f0 0x7ff6580cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6581601f0 0x7ff6580cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:56058/0 (socket says 172.21.15.135:56058) 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 -- 172.21.15.135:0/4179870114 learned_addr learned my addr 172.21.15.135:0/4179870114 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff656ffd640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff65809b100 0x7ff6580b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff64e7fc640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6580cd2b0 0x7ff6580cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 -- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff65809b100 msgr2=0x7ff6580b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff65809b100 0x7ff6580b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 -- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6580cd2b0 msgr2=0x7ff6580cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6580cd2b0 0x7ff6580cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 -- 172.21.15.135:0/4179870114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6580b0db0 con 0x7ff6581601f0 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff656ffd640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff65809b100 0x7ff6580b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff64e7fc640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6580cd2b0 0x7ff6580cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:13.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff6577fe640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6581601f0 0x7ff6580cf610 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff650009f40 tx=0x7ff650004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff64ffff640 1 -- 172.21.15.135:0/4179870114 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650043020 con 0x7ff6581601f0 2024-04-17T06:10:13.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff64ffff640 1 -- 172.21.15.135:0/4179870114 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65003e450 con 0x7ff6581601f0 2024-04-17T06:10:13.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65806d8b0 con 0x7ff6581601f0 2024-04-17T06:10:13.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff64ffff640 1 -- 172.21.15.135:0/4179870114 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650009510 con 0x7ff6581601f0 2024-04-17T06:10:13.924 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.923+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65806d2f0 con 0x7ff6581601f0 2024-04-17T06:10:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.924+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff658068a20 con 0x7ff6581601f0 2024-04-17T06:10:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.925+0000 7ff64ffff640 1 -- 172.21.15.135:0/4179870114 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff650046020 con 0x7ff6581601f0 2024-04-17T06:10:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.925+0000 7ff64ffff640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff63806c8f0 0x7ff63806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.925+0000 7ff656ffd640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff63806c8f0 0x7ff63806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.925+0000 7ff64ffff640 1 -- 172.21.15.135:0/4179870114 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff6500ae820 con 0x7ff6581601f0 2024-04-17T06:10:13.925 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.926+0000 7ff656ffd640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff63806c8f0 0x7ff63806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff64800a990 tx=0x7ff648008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:13.927 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:13.930+0000 7ff64ffff640 1 -- 172.21.15.135:0/4179870114 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff65004d030 con 0x7ff6581601f0 2024-04-17T06:10:14.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.106+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff6580b5030 con 0x7ff6581601f0 2024-04-17T06:10:14.105 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.107+0000 7ff64ffff640 1 -- 172.21.15.135:0/4179870114 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7ff65007c850 con 0x7ff6581601f0 2024-04-17T06:10:14.105 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.108+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff63806c8f0 msgr2=0x7ff63806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.108+0000 7ff65d60d640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff63806c8f0 0x7ff63806edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff64800a990 tx=0x7ff648008040 comp rx=0 tx=0).stop 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.108+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6581601f0 msgr2=0x7ff6580cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.108+0000 7ff65d60d640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6581601f0 0x7ff6580cf610 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff650009f40 tx=0x7ff650004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.109+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 shutdown_connections 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.109+0000 7ff65d60d640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff63806c8f0 0x7ff63806edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.109+0000 7ff65d60d640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6581601f0 0x7ff6580cf610 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.109+0000 7ff65d60d640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6580cd2b0 0x7ff6580cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.107 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.109+0000 7ff65d60d640 1 --2- 172.21.15.135:0/4179870114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff65809b100 0x7ff6580b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.109+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 >> 172.21.15.135:0/4179870114 conn(0x7ff6580b4190 msgr2=0x7ff658070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.109+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 shutdown_connections 2024-04-17T06:10:14.108 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.109+0000 7ff65d60d640 1 -- 172.21.15.135:0/4179870114 wait complete. 2024-04-17T06:10:14.118 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:10:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.226+0000 7f14d8688640 1 Processor -- start 2024-04-17T06:10:14.224 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.226+0000 7f14d8688640 1 -- start start 2024-04-17T06:10:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d8688640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d015ec00 0x7f14d00d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d8688640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d00d3330 con 0x7f14d00d43d0 2024-04-17T06:10:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d8688640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d00d3610 con 0x7f14d00d19e0 2024-04-17T06:10:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d8688640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f14d015efe0 con 0x7f14d015ec00 2024-04-17T06:10:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d5c23640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f14d00d43d0 0x7f14d00d12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:42400/0 (socket says 172.21.15.135:42400) 2024-04-17T06:10:14.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d5c23640 1 -- 172.21.15.135:0/1637528344 learned_addr learned my addr 172.21.15.135:0/1637528344 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:14.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 564803602 0 0) 0x7f14d00d3330 con 0x7f14d00d43d0 2024-04-17T06:10:14.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d6424640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d015ec00 0x7f14d00d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.226 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.227+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14bc003520 con 0x7f14d00d43d0 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1765860029 0 0) 0x7f14bc003520 con 0x7f14d00d43d0 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d015ec00 msgr2=0x7f14d00d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d015ec00 0x7f14d00d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 >> v1:172.21.15.190:6789/0 conn(0x7f14d00d19e0 legacy=0x7f14d00b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d00d38f0 con 0x7f14d00d43d0 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f14c0001670 con 0x7f14d00d43d0 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 >> v1:172.21.15.135:6789/0 conn(0x7f14d00d43d0 legacy=0x7f14d00d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d015ec00 0x7f14bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc003f60 0x7f14bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc004d40 0x7f14bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.229 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14d015efe0 con 0x7f14bc004d40 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc00bb30 con 0x7f14bc003f60 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f14bc00bd30 con 0x7f14d015ec00 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d6c25640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc004d40 0x7f14bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d6424640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d015ec00 0x7f14bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d5c23640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc003f60 0x7f14bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d6c25640 1 -- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d015ec00 msgr2=0x7f14bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d6c25640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d015ec00 0x7f14bc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d6c25640 1 -- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc003f60 msgr2=0x7f14bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d6c25640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc003f60 0x7f14bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.228+0000 7f14d6c25640 1 -- 172.21.15.135:0/1637528344 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc003520 con 0x7f14bc004d40 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.229+0000 7f14d6c25640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc004d40 0x7f14bc0051c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f14c8001720 tx=0x7f14bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5f096775495f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.229+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8001370 con 0x7f14bc004d40 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.229+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c80052d0 con 0x7f14bc004d40 2024-04-17T06:10:14.230 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.229+0000 7f14d5422640 1 -- 172.21.15.135:0/1637528344 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8003830 con 0x7f14bc004d40 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.230+0000 7f14d8688640 1 -- 172.21.15.135:0/1637528344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc004d40 msgr2=0x7f14bc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.230+0000 7f14d8688640 1 --2- 172.21.15.135:0/1637528344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14bc004d40 0x7f14bc0051c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f14c8001720 tx=0x7f14bc005830 comp rx=0 tx=0).stop 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.230+0000 7f14d6c25640 1 -- 172.21.15.135:0/1637528344 reap_dead start 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.230+0000 7f14d8688640 1 -- 172.21.15.135:0/1637528344 shutdown_connections 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.230+0000 7f14d8688640 1 -- 172.21.15.135:0/1637528344 >> 172.21.15.135:0/1637528344 conn(0x7f14d006cc90 msgr2=0x7f14d006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.230+0000 7f14d8688640 1 -- 172.21.15.135:0/1637528344 shutdown_connections 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.230+0000 7f14d8688640 1 -- 172.21.15.135:0/1637528344 wait complete. 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.231+0000 7f14d8688640 1 Processor -- start 2024-04-17T06:10:14.231 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.231+0000 7f14d8688640 1 -- start start 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d8688640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc004d40 0x7f14d00caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d8688640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d014f6e0 0x7f14d00b8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d8688640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d01510b0 0x7f14d00b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d8688640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14d00ba6e0 con 0x7f14d01510b0 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d8688640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f14d00d3f40 con 0x7f14bc004d40 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d8688640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f14d00ba860 con 0x7f14d014f6e0 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d01510b0 0x7f14d00b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d01510b0 0x7f14d00b8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:56078/0 (socket says 172.21.15.135:56078) 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 -- 172.21.15.135:0/2483703790 learned_addr learned my addr 172.21.15.135:0/2483703790 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6424640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc004d40 0x7f14d00caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d5c23640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d014f6e0 0x7f14d00b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.234 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 -- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d014f6e0 msgr2=0x7f14d00b8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d014f6e0 0x7f14d00b8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 -- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc004d40 msgr2=0x7f14d00caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc004d40 0x7f14d00caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 -- 172.21.15.135:0/2483703790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d0150e40 con 0x7f14d01510b0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d5c23640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d014f6e0 0x7f14d00b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d6c25640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d01510b0 0x7f14d00b8ec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f14c802bcb0 tx=0x7f14c8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14c6ffd640 1 -- 172.21.15.135:0/2483703790 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8001370 con 0x7f14d01510b0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14c6ffd640 1 -- 172.21.15.135:0/2483703790 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c8044070 con 0x7f14d01510b0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d014fda0 con 0x7f14d01510b0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14c6ffd640 1 -- 172.21.15.135:0/2483703790 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8007a30 con 0x7f14d01510b0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.232+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d0150360 con 0x7f14d01510b0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.233+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d00d0af0 con 0x7f14d01510b0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.234+0000 7f14c6ffd640 1 -- 172.21.15.135:0/2483703790 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14c803b040 con 0x7f14d01510b0 2024-04-17T06:10:14.235 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.234+0000 7f14c6ffd640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f14b406c920 0x7f14b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.234+0000 7f14c6ffd640 1 -- 172.21.15.135:0/2483703790 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f14c80ad490 con 0x7f14d01510b0 2024-04-17T06:10:14.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.234+0000 7f14d6424640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f14b406c920 0x7f14b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.235+0000 7f14d6424640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f14b406c920 0x7f14b406ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14cc007520 tx=0x7f14cc00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.236 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.239+0000 7f14c6ffd640 1 -- 172.21.15.135:0/2483703790 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14d00d0af0 con 0x7f14d01510b0 2024-04-17T06:10:14.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.411+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14d0074940 con 0x7f14d01510b0 2024-04-17T06:10:14.409 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.412+0000 7f14c6ffd640 1 -- 172.21.15.135:0/2483703790 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f14c807af00 con 0x7f14d01510b0 2024-04-17T06:10:14.410 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:10:14.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f14b406c920 msgr2=0x7f14b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f14b406c920 0x7f14b406ede0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f14cc007520 tx=0x7f14cc00b040 comp rx=0 tx=0).stop 2024-04-17T06:10:14.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d01510b0 msgr2=0x7f14d00b8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d01510b0 0x7f14d00b8ec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f14c802bcb0 tx=0x7f14c8001120 comp rx=0 tx=0).stop 2024-04-17T06:10:14.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 shutdown_connections 2024-04-17T06:10:14.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f14b406c920 0x7f14b406ede0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d01510b0 0x7f14d00b8ec0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d014f6e0 0x7f14d00b8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.413+0000 7f14d8688640 1 --2- 172.21.15.135:0/2483703790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14bc004d40 0x7f14d00caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.414+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 >> 172.21.15.135:0/2483703790 conn(0x7f14d006cc90 msgr2=0x7f14d00b9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:14.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.414+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 shutdown_connections 2024-04-17T06:10:14.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.414+0000 7f14d8688640 1 -- 172.21.15.135:0/2483703790 wait complete. 2024-04-17T06:10:14.422 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-17T06:10:14.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.532+0000 7ff63e2a5640 1 Processor -- start 2024-04-17T06:10:14.530 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.532+0000 7ff63e2a5640 1 -- start start 2024-04-17T06:10:14.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.532+0000 7ff63e2a5640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff6380d43d0 0x7ff6380d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.532+0000 7ff63e2a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6380d1810 0x7ff6380b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.532+0000 7ff63e2a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6380d30a0 con 0x7ff63815ec00 2024-04-17T06:10:14.531 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.532+0000 7ff63e2a5640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff63815efe0 con 0x7ff6380d43d0 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.532+0000 7ff63e2a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff6380d3f40 con 0x7ff6380d1810 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.533+0000 7ff62ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff6380d43d0 0x7ff6380d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.533+0000 7ff62ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff6380d43d0 0x7ff6380d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42180/0 (socket says 172.21.15.135:42180) 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.533+0000 7ff62ffff640 1 -- 172.21.15.135:0/1922588434 learned_addr learned my addr 172.21.15.135:0/1922588434 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.533+0000 7ff63c842640 1 --2- 172.21.15.135:0/1922588434 >> v2:172.21.15.190:3300/0 conn(0x7ff6380d1810 0x7ff6380b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.533+0000 7ff62ffff640 1 -- 172.21.15.135:0/1922588434 >> v2:172.21.15.190:3300/0 conn(0x7ff6380d1810 msgr2=0x7ff6380b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.533+0000 7ff62ffff640 1 --2- 172.21.15.135:0/1922588434 >> v2:172.21.15.190:3300/0 conn(0x7ff6380d1810 0x7ff6380b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.533+0000 7ff62ffff640 1 -- 172.21.15.135:0/1922588434 >> v1:172.21.15.190:6789/0 conn(0x7ff63815ec00 legacy=0x7ff6380d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.533+0000 7ff62ffff640 1 -- 172.21.15.135:0/1922588434 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6380d3330 con 0x7ff6380d43d0 2024-04-17T06:10:14.532 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff62ffff640 1 --2- 172.21.15.135:0/1922588434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d43d0 0x7ff6380d12d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff63806dab0 tx=0x7ff62002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb213a13588c514 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.533 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff6377fe640 1 -- 172.21.15.135:0/1922588434 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62002f850 con 0x7ff6380d43d0 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff6377fe640 1 -- 172.21.15.135:0/1922588434 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62002f9f0 con 0x7ff6380d43d0 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff6377fe640 1 -- 172.21.15.135:0/1922588434 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62002fdc0 con 0x7ff6380d43d0 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff63e2a5640 1 -- 172.21.15.135:0/1922588434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d43d0 msgr2=0x7ff6380d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/1922588434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d43d0 0x7ff6380d12d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff63806dab0 tx=0x7ff62002ba30 comp rx=0 tx=0).stop 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff63e2a5640 1 -- 172.21.15.135:0/1922588434 shutdown_connections 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/1922588434 >> v2:172.21.15.190:3300/0 conn(0x7ff6380d1810 0x7ff6380b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/1922588434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d43d0 0x7ff6380d12d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.534+0000 7ff63e2a5640 1 -- 172.21.15.135:0/1922588434 >> 172.21.15.135:0/1922588434 conn(0x7ff63806cc90 msgr2=0x7ff63806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.535+0000 7ff63e2a5640 1 -- 172.21.15.135:0/1922588434 shutdown_connections 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.535+0000 7ff63e2a5640 1 -- 172.21.15.135:0/1922588434 wait complete. 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff63e2a5640 1 Processor -- start 2024-04-17T06:10:14.534 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff63e2a5640 1 -- start start 2024-04-17T06:10:14.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff63e2a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d1810 0x7ff638154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.536 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff63e2a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6380d43d0 0x7ff6380d4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff63e2a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff63815ec00 0x7ff6380d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff63e2a5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380d38f0 con 0x7ff6380d1810 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff63e2a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff6380d1c60 con 0x7ff63815ec00 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff63e2a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff6380d3bd0 con 0x7ff6380d43d0 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.536+0000 7ff637fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d1810 0x7ff638154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff637fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d1810 0x7ff638154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42192/0 (socket says 172.21.15.135:42192) 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff637fff640 1 -- 172.21.15.135:0/3360728491 learned_addr learned my addr 172.21.15.135:0/3360728491 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff63c842640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff63815ec00 0x7ff6380d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff62ffff640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6380d43d0 0x7ff6380d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff637fff640 1 -- 172.21.15.135:0/3360728491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6380d43d0 msgr2=0x7ff6380d4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff637fff640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6380d43d0 0x7ff6380d4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff637fff640 1 -- 172.21.15.135:0/3360728491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff63815ec00 msgr2=0x7ff6380d4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff637fff640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff63815ec00 0x7ff6380d4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff637fff640 1 -- 172.21.15.135:0/3360728491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6380cc8f0 con 0x7ff6380d1810 2024-04-17T06:10:14.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff637fff640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d1810 0x7ff638154eb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff62800c3f0 tx=0x7ff62800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff6357fa640 1 -- 172.21.15.135:0/3360728491 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280052d0 con 0x7ff6380d1810 2024-04-17T06:10:14.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff6357fa640 1 -- 172.21.15.135:0/3360728491 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff628005490 con 0x7ff6380d1810 2024-04-17T06:10:14.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff638161100 con 0x7ff6380d1810 2024-04-17T06:10:14.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff6357fa640 1 -- 172.21.15.135:0/3360728491 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628007260 con 0x7ff6380d1810 2024-04-17T06:10:14.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.537+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6381616c0 con 0x7ff6380d1810 2024-04-17T06:10:14.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.538+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff638074940 con 0x7ff6380d1810 2024-04-17T06:10:14.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.539+0000 7ff6357fa640 1 -- 172.21.15.135:0/3360728491 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff628007400 con 0x7ff6380d1810 2024-04-17T06:10:14.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.539+0000 7ff6357fa640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff61806c8f0 0x7ff61806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.539+0000 7ff6357fa640 1 -- 172.21.15.135:0/3360728491 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7ff62808b360 con 0x7ff6380d1810 2024-04-17T06:10:14.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.539+0000 7ff62ffff640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff61806c8f0 0x7ff61806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.540+0000 7ff62ffff640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff61806c8f0 0x7ff61806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff620036000 tx=0x7ff620008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.543+0000 7ff6357fa640 1 -- 172.21.15.135:0/3360728491 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6380cc8f0 con 0x7ff6380d1810 2024-04-17T06:10:14.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.719+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff6380b8810 con 0x7ff6380d1810 2024-04-17T06:10:14.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.719+0000 7ff6357fa640 1 -- 172.21.15.135:0/3360728491 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7ff628059420 con 0x7ff6380d1810 2024-04-17T06:10:14.717 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:10:14.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff61806c8f0 msgr2=0x7ff61806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff61806c8f0 0x7ff61806edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff620036000 tx=0x7ff620008030 comp rx=0 tx=0).stop 2024-04-17T06:10:14.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d1810 msgr2=0x7ff638154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d1810 0x7ff638154eb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff62800c3f0 tx=0x7ff62800c8b0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 shutdown_connections 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff63815ec00 0x7ff6380d4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6380d43d0 0x7ff6380d4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff61806c8f0 0x7ff61806edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 --2- 172.21.15.135:0/3360728491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6380d1810 0x7ff638154eb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 >> 172.21.15.135:0/3360728491 conn(0x7ff63806cc90 msgr2=0x7ff6380b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 shutdown_connections 2024-04-17T06:10:14.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.721+0000 7ff63e2a5640 1 -- 172.21.15.135:0/3360728491 wait complete. 2024-04-17T06:10:14.732 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:10:14.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.837+0000 7f81e6511640 1 Processor -- start 2024-04-17T06:10:14.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.837+0000 7f81e6511640 1 -- start start 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81e6511640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f81e015bb00 0x7f81e00b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81e6511640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00b3590 0x7f81e00af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81e6511640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81e00b3a50 con 0x7f81e00afc00 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81e6511640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f81e00b0080 con 0x7f81e015bb00 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81e6511640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f81e00b4660 con 0x7f81e00b3590 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81e4aae640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f81e00afc00 0x7f81e0071ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46336/0 (socket says 172.21.15.135:46336) 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81e4aae640 1 -- 172.21.15.135:0/718551045 learned_addr learned my addr 172.21.15.135:0/718551045 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81dffff640 1 --2- 172.21.15.135:0/718551045 >> v2:172.21.15.190:3300/0 conn(0x7f81e015bb00 0x7f81e00b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81df7fe640 1 --2- 172.21.15.135:0/718551045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00b3590 0x7f81e00af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81deffd640 1 -- 172.21.15.135:0/718551045 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731374447 0 0) 0x7f81e00b3a50 con 0x7f81e00afc00 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81deffd640 1 -- 172.21.15.135:0/718551045 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81c8003520 con 0x7f81e00afc00 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81df7fe640 1 -- 172.21.15.135:0/718551045 >> v2:172.21.15.190:3300/0 conn(0x7f81e015bb00 msgr2=0x7f81e00b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:14.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81df7fe640 1 --2- 172.21.15.135:0/718551045 >> v2:172.21.15.190:3300/0 conn(0x7f81e015bb00 0x7f81e00b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81df7fe640 1 -- 172.21.15.135:0/718551045 >> v1:172.21.15.135:6789/0 conn(0x7f81e00afc00 legacy=0x7f81e0071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.838+0000 7f81df7fe640 1 -- 172.21.15.135:0/718551045 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e00b3d30 con 0x7f81e00b3590 2024-04-17T06:10:14.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.839+0000 7f81df7fe640 1 --2- 172.21.15.135:0/718551045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00b3590 0x7f81e00af6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81d40097a0 tx=0x7f81d402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1431393406807b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.839+0000 7f81deffd640 1 -- 172.21.15.135:0/718551045 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4035020 con 0x7f81e00b3590 2024-04-17T06:10:14.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.839+0000 7f81deffd640 1 -- 172.21.15.135:0/718551045 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d4031830 con 0x7f81e00b3590 2024-04-17T06:10:14.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.839+0000 7f81deffd640 1 -- 172.21.15.135:0/718551045 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4031c00 con 0x7f81e00b3590 2024-04-17T06:10:14.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.840+0000 7f81e6511640 1 -- 172.21.15.135:0/718551045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00b3590 msgr2=0x7f81e00af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.840+0000 7f81e6511640 1 --2- 172.21.15.135:0/718551045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00b3590 0x7f81e00af6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81d40097a0 tx=0x7f81d402b8d0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.840+0000 7f81e6511640 1 -- 172.21.15.135:0/718551045 shutdown_connections 2024-04-17T06:10:14.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.840+0000 7f81e6511640 1 --2- 172.21.15.135:0/718551045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00b3590 0x7f81e00af6c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.840+0000 7f81e6511640 1 --2- 172.21.15.135:0/718551045 >> v2:172.21.15.190:3300/0 conn(0x7f81e015bb00 0x7f81e00b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.840+0000 7f81e6511640 1 -- 172.21.15.135:0/718551045 >> 172.21.15.135:0/718551045 conn(0x7f81e00d0520 msgr2=0x7f81e00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:14.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.840+0000 7f81e6511640 1 -- 172.21.15.135:0/718551045 shutdown_connections 2024-04-17T06:10:14.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.840+0000 7f81e6511640 1 -- 172.21.15.135:0/718551045 wait complete. 2024-04-17T06:10:14.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.841+0000 7f81e6511640 1 Processor -- start 2024-04-17T06:10:14.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.841+0000 7f81e6511640 1 -- start start 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e6511640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00afc00 0x7f81e0146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e6511640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81e00b3590 0x7f81e00a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e6511640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81e015bb00 0x7f81e006ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e6511640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81e00b5690 con 0x7f81e015bb00 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e6511640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f81e00b42f0 con 0x7f81e00b3590 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e6511640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f81e00b5810 con 0x7f81e00afc00 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e4aae640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81e015bb00 0x7f81e006ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e4aae640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81e015bb00 0x7f81e006ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42194/0 (socket says 172.21.15.135:42194) 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e4aae640 1 -- 172.21.15.135:0/772218318 learned_addr learned my addr 172.21.15.135:0/772218318 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81dffff640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00afc00 0x7f81e0146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81df7fe640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81e00b3590 0x7f81e00a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e4aae640 1 -- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00afc00 msgr2=0x7f81e0146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e4aae640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00afc00 0x7f81e0146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e4aae640 1 -- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81e00b3590 msgr2=0x7f81e00a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.842+0000 7f81e4aae640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81e00b3590 0x7f81e00a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.843+0000 7f81e4aae640 1 -- 172.21.15.135:0/772218318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e01474f0 con 0x7f81e015bb00 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.843+0000 7f81df7fe640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81e00b3590 0x7f81e00a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.843+0000 7f81e4aae640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81e015bb00 0x7f81e006ef70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f81d800cb70 tx=0x7f81d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.843+0000 7f81dcff9640 1 -- 172.21.15.135:0/772218318 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8017070 con 0x7f81e015bb00 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.843+0000 7f81dcff9640 1 -- 172.21.15.135:0/772218318 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d800ad40 con 0x7f81e015bb00 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.843+0000 7f81dcff9640 1 -- 172.21.15.135:0/772218318 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8012620 con 0x7f81e015bb00 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.843+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e01f3150 con 0x7f81e015bb00 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.843+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e01f3710 con 0x7f81e015bb00 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.844+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e0074940 con 0x7f81e015bb00 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.845+0000 7f81dcff9640 1 -- 172.21.15.135:0/772218318 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81d801a070 con 0x7f81e015bb00 2024-04-17T06:10:14.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.845+0000 7f81dcff9640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81cc06c920 0x7f81cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:14.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.845+0000 7f81dcff9640 1 -- 172.21.15.135:0/772218318 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f81d808bd40 con 0x7f81e015bb00 2024-04-17T06:10:14.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.845+0000 7f81dffff640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81cc06c920 0x7f81cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:14.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.845+0000 7f81dffff640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81cc06c920 0x7f81cc06ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81e014ab00 tx=0x7f81d0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:14.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:14.849+0000 7f81dcff9640 1 -- 172.21.15.135:0/772218318 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81e01474f0 con 0x7f81e015bb00 2024-04-17T06:10:15.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.064+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81e006ce80 con 0x7f81e015bb00 2024-04-17T06:10:15.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.065+0000 7f81dcff9640 1 -- 172.21.15.135:0/772218318 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f81d8059d70 con 0x7f81e015bb00 2024-04-17T06:10:15.063 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:15.063 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:36.038772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:10:15.063 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 9 2024-04-17T06:10:15.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81cc06c920 msgr2=0x7f81cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81cc06c920 0x7f81cc06ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81e014ab00 tx=0x7f81d0008040 comp rx=0 tx=0).stop 2024-04-17T06:10:15.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81e015bb00 msgr2=0x7f81e006ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81e015bb00 0x7f81e006ef70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f81d800cb70 tx=0x7f81d800a430 comp rx=0 tx=0).stop 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 shutdown_connections 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f81cc06c920 0x7f81cc06ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81e015bb00 0x7f81e006ef70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81e00b3590 0x7f81e00a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 --2- 172.21.15.135:0/772218318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81e00afc00 0x7f81e0146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 >> 172.21.15.135:0/772218318 conn(0x7f81e00d0520 msgr2=0x7f81e00cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 shutdown_connections 2024-04-17T06:10:15.066 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.067+0000 7f81e6511640 1 -- 172.21.15.135:0/772218318 wait complete. 2024-04-17T06:10:15.077 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:15.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.186+0000 7f0503a63640 1 Processor -- start 2024-04-17T06:10:15.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.187+0000 7f0503a63640 1 -- start start 2024-04-17T06:10:15.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0503a63640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04fc15ed00 0x7f04fc0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0503a63640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f04fc09bd50 0x7f04fc09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0503a63640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04fc0d13f0 con 0x7f04fc09af30 2024-04-17T06:10:15.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0503a63640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f04fc0d19b0 con 0x7f04fc09bd50 2024-04-17T06:10:15.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0503a63640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f04fc0d16d0 con 0x7f04fc15ed00 2024-04-17T06:10:15.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0500ffe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f04fc09bd50 0x7f04fc09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0500ffe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f04fc09bd50 0x7f04fc09a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42204/0 (socket says 172.21.15.135:42204) 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0500ffe640 1 -- 172.21.15.135:0/2519370306 learned_addr learned my addr 172.21.15.135:0/2519370306 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f05017ff640 1 --2- 172.21.15.135:0/2519370306 >> v2:172.21.15.190:3300/0 conn(0x7f04fc15ed00 0x7f04fc0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0500ffe640 1 -- 172.21.15.135:0/2519370306 >> v2:172.21.15.190:3300/0 conn(0x7f04fc15ed00 msgr2=0x7f04fc0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0500ffe640 1 --2- 172.21.15.135:0/2519370306 >> v2:172.21.15.190:3300/0 conn(0x7f04fc15ed00 0x7f04fc0d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0500ffe640 1 -- 172.21.15.135:0/2519370306 >> v1:172.21.15.190:6789/0 conn(0x7f04fc09af30 legacy=0x7f04fc0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.188+0000 7f0500ffe640 1 -- 172.21.15.135:0/2519370306 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc06cc20 con 0x7f04fc09bd50 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.189+0000 7f0500ffe640 1 --2- 172.21.15.135:0/2519370306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09bd50 0x7f04fc09a9f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f04fc073aa0 tx=0x7f04f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b332c18ec80418 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.189+0000 7f04fbfff640 1 -- 172.21.15.135:0/2519370306 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0035020 con 0x7f04fc09bd50 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.189+0000 7f04fbfff640 1 -- 172.21.15.135:0/2519370306 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f0031870 con 0x7f04fc09bd50 2024-04-17T06:10:15.188 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.189+0000 7f04fbfff640 1 -- 172.21.15.135:0/2519370306 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0031c40 con 0x7f04fc09bd50 2024-04-17T06:10:15.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.190+0000 7f0503a63640 1 -- 172.21.15.135:0/2519370306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09bd50 msgr2=0x7f04fc09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.190+0000 7f0503a63640 1 --2- 172.21.15.135:0/2519370306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09bd50 0x7f04fc09a9f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f04fc073aa0 tx=0x7f04f002b910 comp rx=0 tx=0).stop 2024-04-17T06:10:15.191 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.190+0000 7f0503a63640 1 -- 172.21.15.135:0/2519370306 shutdown_connections 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.190+0000 7f0503a63640 1 --2- 172.21.15.135:0/2519370306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09bd50 0x7f04fc09a9f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.190+0000 7f0503a63640 1 --2- 172.21.15.135:0/2519370306 >> v2:172.21.15.190:3300/0 conn(0x7f04fc15ed00 0x7f04fc0d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.190+0000 7f0503a63640 1 -- 172.21.15.135:0/2519370306 >> 172.21.15.135:0/2519370306 conn(0x7f04fc0b4190 msgr2=0x7f04fc0b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.190+0000 7f0503a63640 1 -- 172.21.15.135:0/2519370306 shutdown_connections 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.190+0000 7f0503a63640 1 -- 172.21.15.135:0/2519370306 wait complete. 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.191+0000 7f0503a63640 1 Processor -- start 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.191+0000 7f0503a63640 1 -- start start 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0503a63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09af30 0x7f04fc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0503a63640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04fc09bd50 0x7f04fc15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.192 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0503a63640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04fc15ed00 0x7f04fc0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0503a63640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc06e430 con 0x7f04fc09af30 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0503a63640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc0d10e0 con 0x7f04fc09bd50 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0503a63640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc06e5b0 con 0x7f04fc15ed00 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09af30 0x7f04fc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09af30 0x7f04fc0cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42206/0 (socket says 172.21.15.135:42206) 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 -- 172.21.15.135:0/606704536 learned_addr learned my addr 172.21.15.135:0/606704536 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0500ffe640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04fc09bd50 0x7f04fc15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 -- 172.21.15.135:0/606704536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04fc15ed00 msgr2=0x7f04fc0b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04fc15ed00 0x7f04fc0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 -- 172.21.15.135:0/606704536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04fc09bd50 msgr2=0x7f04fc15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04fc09bd50 0x7f04fc15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 -- 172.21.15.135:0/606704536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc1f4860 con 0x7f04fc09af30 2024-04-17T06:10:15.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0500ffe640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04fc09bd50 0x7f04fc15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:15.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f05017ff640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09af30 0x7f04fc0cc160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00ccd0 tx=0x7f04ec00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f04f9ffb640 1 -- 172.21.15.135:0/606704536 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec003990 con 0x7f04fc09af30 2024-04-17T06:10:15.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f04f9ffb640 1 -- 172.21.15.135:0/606704536 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec003b30 con 0x7f04fc09af30 2024-04-17T06:10:15.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.192+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc1f4b40 con 0x7f04fc09af30 2024-04-17T06:10:15.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.193+0000 7f04f9ffb640 1 -- 172.21.15.135:0/606704536 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec004020 con 0x7f04fc09af30 2024-04-17T06:10:15.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.193+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc1f5100 con 0x7f04fc09af30 2024-04-17T06:10:15.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.194+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc1f5340 con 0x7f04fc09af30 2024-04-17T06:10:15.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.194+0000 7f04f9ffb640 1 -- 172.21.15.135:0/606704536 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04ec0041c0 con 0x7f04fc09af30 2024-04-17T06:10:15.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.195+0000 7f04f9ffb640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.195+0000 7f04f9ffb640 1 -- 172.21.15.135:0/606704536 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f04ec08c340 con 0x7f04fc09af30 2024-04-17T06:10:15.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.195+0000 7f0500ffe640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.195 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.195+0000 7f0500ffe640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f04e806c8f0 0x7f04e806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04f0036000 tx=0x7f04f0008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.199+0000 7f04f9ffb640 1 -- 172.21.15.135:0/606704536 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04ec014030 con 0x7f04fc09af30 2024-04-17T06:10:15.374 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.376+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04fc068a20 con 0x7f04fc09af30 2024-04-17T06:10:15.375 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.377+0000 7f04f9ffb640 1 -- 172.21.15.135:0/606704536 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18950 (secure 0 0 0) 0x7f04ec05a400 con 0x7f04fc09af30 2024-04-17T06:10:15.375 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:15.375 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:01.008924+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:15.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f04e806c8f0 msgr2=0x7f04e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f04e806c8f0 0x7f04e806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04f0036000 tx=0x7f04f0008030 comp rx=0 tx=0).stop 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09af30 msgr2=0x7f04fc0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09af30 0x7f04fc0cc160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00ccd0 tx=0x7f04ec00a5c0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 shutdown_connections 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04fc15ed00 0x7f04fc0b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04fc09bd50 0x7f04fc15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 --2- 172.21.15.135:0/606704536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04fc09af30 0x7f04fc0cc160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 >> 172.21.15.135:0/606704536 conn(0x7f04fc0b4190 msgr2=0x7f04fc0b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 shutdown_connections 2024-04-17T06:10:15.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.379+0000 7f0503a63640 1 -- 172.21.15.135:0/606704536 wait complete. 2024-04-17T06:10:15.389 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-17T06:10:15.389 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:10:15.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.494+0000 7f1ac3055640 1 Processor -- start 2024-04-17T06:10:15.492 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.494+0000 7f1ac3055640 1 -- start start 2024-04-17T06:10:15.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac3055640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 0x7f1abc06b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac3055640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1abc0cfe10 0x7f1abc15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac3055640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1abc0b5ae0 con 0x7f1abc0d0630 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac3055640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f1abc0b60a0 con 0x7f1abc0cfe10 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac3055640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc0b5dc0 con 0x7f1abc15ec70 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac15f2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1abc0cfe10 0x7f1abc15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac15f2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f1abc0cfe10 0x7f1abc15f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42216/0 (socket says 172.21.15.135:42216) 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac15f2640 1 -- 172.21.15.135:0/3540414311 learned_addr learned my addr 172.21.15.135:0/3540414311 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac0df1640 1 --2- 172.21.15.135:0/3540414311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 0x7f1abc06b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.495+0000 7f1ac15f2640 1 -- 172.21.15.135:0/3540414311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 msgr2=0x7f1abc06b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.496+0000 7f1ac15f2640 1 --2- 172.21.15.135:0/3540414311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 0x7f1abc06b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.496+0000 7f1ac15f2640 1 -- 172.21.15.135:0/3540414311 >> v1:172.21.15.190:6789/0 conn(0x7f1abc0d0630 legacy=0x7f1abc0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.496+0000 7f1ac15f2640 1 -- 172.21.15.135:0/3540414311 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc0b3e80 con 0x7f1abc0cfe10 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.496+0000 7f1ac15f2640 1 --2- 172.21.15.135:0/3540414311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15f050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009e50 tx=0x7f1ab402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b4f468ab6378f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.496+0000 7f1abb7fe640 1 -- 172.21.15.135:0/3540414311 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab402ca70 con 0x7f1abc0cfe10 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.496+0000 7f1abb7fe640 1 -- 172.21.15.135:0/3540414311 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab402cc10 con 0x7f1abc0cfe10 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.496+0000 7f1abb7fe640 1 -- 172.21.15.135:0/3540414311 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40343f0 con 0x7f1abc0cfe10 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.497+0000 7f1ac3055640 1 -- 172.21.15.135:0/3540414311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 msgr2=0x7f1abc15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.497+0000 7f1ac3055640 1 --2- 172.21.15.135:0/3540414311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15f050 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009e50 tx=0x7f1ab402bed0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.497+0000 7f1ac3055640 1 -- 172.21.15.135:0/3540414311 shutdown_connections 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.497+0000 7f1ac3055640 1 --2- 172.21.15.135:0/3540414311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15f050 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.497+0000 7f1ac3055640 1 --2- 172.21.15.135:0/3540414311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 0x7f1abc06b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.497+0000 7f1ac3055640 1 -- 172.21.15.135:0/3540414311 >> 172.21.15.135:0/3540414311 conn(0x7f1abc0cd7e0 msgr2=0x7f1abc0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.497+0000 7f1ac3055640 1 -- 172.21.15.135:0/3540414311 shutdown_connections 2024-04-17T06:10:15.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.497+0000 7f1ac3055640 1 -- 172.21.15.135:0/3540414311 wait complete. 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.498+0000 7f1ac3055640 1 Processor -- start 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.498+0000 7f1ac3055640 1 -- start start 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac3055640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac3055640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1abc0d0630 0x7f1abc15f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac3055640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 0x7f1abc1613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac3055640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0d02a0 con 0x7f1abc0cfe10 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac3055640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0d0ab0 con 0x7f1abc0d0630 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac3055640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc0d0420 con 0x7f1abc15ec70 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac0df1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac0df1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42230/0 (socket says 172.21.15.135:42230) 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac0df1640 1 -- 172.21.15.135:0/2409946942 learned_addr learned my addr 172.21.15.135:0/2409946942 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac15f2640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 0x7f1abc1613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1abbfff640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1abc0d0630 0x7f1abc15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac0df1640 1 -- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 msgr2=0x7f1abc1613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac0df1640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 0x7f1abc1613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac0df1640 1 -- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1abc0d0630 msgr2=0x7f1abc15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac0df1640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1abc0d0630 0x7f1abc15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1ac0df1640 1 -- 172.21.15.135:0/2409946942 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc15dd70 con 0x7f1abc0cfe10 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.499+0000 7f1abbfff640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1abc0d0630 0x7f1abc15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.500+0000 7f1ac0df1640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15e100 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e870 tx=0x7f1aac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.500+0000 7f1ab97fa640 1 -- 172.21.15.135:0/2409946942 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac017070 con 0x7f1abc0cfe10 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.500+0000 7f1ab97fa640 1 -- 172.21.15.135:0/2409946942 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac00c940 con 0x7f1abc0cfe10 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.500+0000 7f1ab97fa640 1 -- 172.21.15.135:0/2409946942 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00cd30 con 0x7f1abc0cfe10 2024-04-17T06:10:15.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.500+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc15f6e0 con 0x7f1abc0cfe10 2024-04-17T06:10:15.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.500+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc15fca0 con 0x7f1abc0cfe10 2024-04-17T06:10:15.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.501+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc1603c0 con 0x7f1abc0cfe10 2024-04-17T06:10:15.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.501+0000 7f1ab97fa640 1 -- 172.21.15.135:0/2409946942 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1aac01a070 con 0x7f1abc0cfe10 2024-04-17T06:10:15.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.502+0000 7f1ab97fa640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.502+0000 7f1ab97fa640 1 -- 172.21.15.135:0/2409946942 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1aac08be20 con 0x7f1abc0cfe10 2024-04-17T06:10:15.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.502+0000 7f1abbfff640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.502 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.502+0000 7f1abbfff640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1aa806c8f0 0x7f1aa806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1abc15d7c0 tx=0x7f1ab0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.506+0000 7f1ab97fa640 1 -- 172.21.15.135:0/2409946942 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1abc15dd70 con 0x7f1abc0cfe10 2024-04-17T06:10:15.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.719+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1abc074940 con 0x7f1abc0cfe10 2024-04-17T06:10:15.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.719+0000 7f1ab97fa640 1 -- 172.21.15.135:0/2409946942 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f1aac059e50 con 0x7f1abc0cfe10 2024-04-17T06:10:15.718 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:15.719 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:36.038772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:10:15.720 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 9 2024-04-17T06:10:15.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1aa806c8f0 msgr2=0x7f1aa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1aa806c8f0 0x7f1aa806edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1abc15d7c0 tx=0x7f1ab0009040 comp rx=0 tx=0).stop 2024-04-17T06:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 msgr2=0x7f1abc15e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.722 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15e100 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e870 tx=0x7f1aac00ed30 comp rx=0 tx=0).stop 2024-04-17T06:10:15.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 shutdown_connections 2024-04-17T06:10:15.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1abc15ec70 0x7f1abc1613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1abc0d0630 0x7f1abc15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f1aa806c8f0 0x7f1aa806edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 --2- 172.21.15.135:0/2409946942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1abc0cfe10 0x7f1abc15e100 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.723 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 >> 172.21.15.135:0/2409946942 conn(0x7f1abc0cd7e0 msgr2=0x7f1abc0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:15.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 shutdown_connections 2024-04-17T06:10:15.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.721+0000 7f1ac3055640 1 -- 172.21.15.135:0/2409946942 wait complete. 2024-04-17T06:10:15.732 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.838+0000 7f7cb5e47640 1 Processor -- start 2024-04-17T06:10:15.836 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.838+0000 7f7cb5e47640 1 -- start start 2024-04-17T06:10:15.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.838+0000 7f7cb5e47640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7cb0158a50 0x7f7cb015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7cb5e47640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb015fb80 con 0x7f7cb0158e30 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7cb5e47640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb015f2e0 con 0x7f7cb006f610 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7cb5e47640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb015b840 con 0x7f7cb0158a50 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7ca7fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7cb0158e30 0x7f7cb00b3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46350/0 (socket says 172.21.15.135:46350) 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7ca7fff640 1 -- 172.21.15.135:0/2100596601 learned_addr learned my addr 172.21.15.135:0/2100596601 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caf7fe640 1 --2- 172.21.15.135:0/2100596601 >> v2:172.21.15.190:3300/0 conn(0x7f7cb0158a50 0x7f7cb015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153124192 0 0) 0x7f7cb015fb80 con 0x7f7cb0158e30 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c9c003520 con 0x7f7cb0158e30 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2603858295 0 0) 0x7f7c9c003520 con 0x7f7cb0158e30 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 >> v2:172.21.15.190:3300/0 conn(0x7f7cb0158a50 msgr2=0x7f7cb015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 --2- 172.21.15.135:0/2100596601 >> v2:172.21.15.190:3300/0 conn(0x7f7cb0158a50 0x7f7cb015ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 >> v1:172.21.15.190:6789/0 conn(0x7f7cb006f610 legacy=0x7f7cb015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb015f5c0 con 0x7f7cb0158e30 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f7c98001670 con 0x7f7cb0158e30 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 >> v1:172.21.15.135:6789/0 conn(0x7f7cb0158e30 legacy=0x7f7cb00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c9c0039b0 0x7f7c9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c004800 0x7f7c9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c0051c0 0x7f7c9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb015b840 con 0x7f7c9c004800 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c00c060 con 0x7f7c9c0051c0 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c00c260 con 0x7f7c9c0039b0 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7ca7fff640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c004800 0x7f7c9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caf7fe640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c9c0039b0 0x7f7c9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.839+0000 7f7caffff640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c0051c0 0x7f7c9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.840+0000 7f7ca7fff640 1 -- 172.21.15.135:0/2100596601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c9c0039b0 msgr2=0x7f7c9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.840+0000 7f7ca7fff640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c9c0039b0 0x7f7c9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.840+0000 7f7ca7fff640 1 -- 172.21.15.135:0/2100596601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c0051c0 msgr2=0x7f7c9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.840+0000 7f7ca7fff640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c0051c0 0x7f7c9c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.840+0000 7f7caffff640 1 -- 172.21.15.135:0/2100596601 reap_dead start 2024-04-17T06:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.840+0000 7f7ca7fff640 1 -- 172.21.15.135:0/2100596601 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c003520 con 0x7f7c9c004800 2024-04-17T06:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.840+0000 7f7ca7fff640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c004800 0x7f7c9c004c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c240 tx=0x7f7c9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39f216ada1d35f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98007cf0 con 0x7f7c9c004800 2024-04-17T06:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c98030070 con 0x7f7c9c004800 2024-04-17T06:10:15.841 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7caeffd640 1 -- 172.21.15.135:0/2100596601 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98005d30 con 0x7f7c9c004800 2024-04-17T06:10:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2100596601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c004800 msgr2=0x7f7c9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7cb5e47640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c004800 0x7f7c9c004c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c240 tx=0x7f7c9c005830 comp rx=0 tx=0).stop 2024-04-17T06:10:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2100596601 shutdown_connections 2024-04-17T06:10:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7cb5e47640 1 --2- 172.21.15.135:0/2100596601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7c9c004800 0x7f7c9c004c80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2100596601 >> 172.21.15.135:0/2100596601 conn(0x7f7cb00d0520 msgr2=0x7f7cb00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2100596601 shutdown_connections 2024-04-17T06:10:15.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.841+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2100596601 wait complete. 2024-04-17T06:10:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.842+0000 7f7cb5e47640 1 Processor -- start 2024-04-17T06:10:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7cb5e47640 1 -- start start 2024-04-17T06:10:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7cb5e47640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c004800 0x7f7cb014f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7cb5e47640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cb0151740 0x7f7cb00ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7cb5e47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb00cea70 0x7f7cb0155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7cb5e47640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb015ec20 con 0x7f7cb00cea70 2024-04-17T06:10:15.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7cb5e47640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb006faa0 con 0x7f7c9c004800 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7cb5e47640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb015eda0 con 0x7f7cb0151740 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7caffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb00cea70 0x7f7cb0155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7caffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb00cea70 0x7f7cb0155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42238/0 (socket says 172.21.15.135:42238) 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7caffff640 1 -- 172.21.15.135:0/2124260791 learned_addr learned my addr 172.21.15.135:0/2124260791 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7ca7fff640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cb0151740 0x7f7cb00ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.843+0000 7f7caf7fe640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c004800 0x7f7cb014f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7caffff640 1 -- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cb0151740 msgr2=0x7f7cb00ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7caffff640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cb0151740 0x7f7cb00ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7caffff640 1 -- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c004800 msgr2=0x7f7cb014f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7caffff640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c004800 0x7f7cb014f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7caffff640 1 -- 172.21.15.135:0/2124260791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb01500e0 con 0x7f7cb00cea70 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7ca7fff640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cb0151740 0x7f7cb00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:15.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7caffff640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb00cea70 0x7f7cb0155420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00cd1c0 tx=0x7f7ca800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7cacff9640 1 -- 172.21.15.135:0/2124260791 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca801a070 con 0x7f7cb00cea70 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7cacff9640 1 -- 172.21.15.135:0/2124260791 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca8006d60 con 0x7f7cb00cea70 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb0156000 con 0x7f7cb00cea70 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7cacff9640 1 -- 172.21.15.135:0/2124260791 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8003bb0 con 0x7f7cb00cea70 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.844+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb0150c60 con 0x7f7cb00cea70 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.845+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb0074940 con 0x7f7cb00cea70 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.845+0000 7f7cacff9640 1 -- 172.21.15.135:0/2124260791 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ca8005070 con 0x7f7cb00cea70 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.846+0000 7f7cacff9640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c9006c920 0x7f7c9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.846+0000 7f7cacff9640 1 -- 172.21.15.135:0/2124260791 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7ca808b250 con 0x7f7cb00cea70 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.846+0000 7f7caf7fe640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c9006c920 0x7f7c9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:15.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.846+0000 7f7caf7fe640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c9006c920 0x7f7c9006ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000ac80 tx=0x7f7ca0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:15.848 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:15.850+0000 7f7cacff9640 1 -- 172.21.15.135:0/2124260791 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cb01500e0 con 0x7f7cb00cea70 2024-04-17T06:10:16.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.062+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cb00d0210 con 0x7f7cb00cea70 2024-04-17T06:10:16.060 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.062+0000 7f7cacff9640 1 -- 172.21.15.135:0/2124260791 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f7ca8059360 con 0x7f7cb00cea70 2024-04-17T06:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:16.061 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/3346397037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3346397037},{"type":"v1","addr":"172.21.15.135:6836","nonce":3346397037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/67768503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":67768503},{"type":"v1","addr":"172.21.15.135:6837","nonce":67768503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2918496251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2918496251},{"type":"v1","addr":"172.21.15.190:6833","nonce":2918496251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:09:19.967271+0000","modified":"2024-04-17T06:09:36.038772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3795516465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":3795516465},{"type":"v1","addr":"172.21.15.190:6835","nonce":3795516465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-17T06:10:16.061 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 9 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c9006c920 msgr2=0x7f7c9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c9006c920 0x7f7c9006ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000ac80 tx=0x7f7ca0008040 comp rx=0 tx=0).stop 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb00cea70 msgr2=0x7f7cb0155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb00cea70 0x7f7cb0155420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7cb00cd1c0 tx=0x7f7ca800fc00 comp rx=0 tx=0).stop 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 shutdown_connections 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7c9006c920 0x7f7c9006ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb00cea70 0x7f7cb0155420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7cb0151740 0x7f7cb00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 --2- 172.21.15.135:0/2124260791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c9c004800 0x7f7cb014f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 >> 172.21.15.135:0/2124260791 conn(0x7f7cb00d0520 msgr2=0x7f7cb006df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 shutdown_connections 2024-04-17T06:10:16.064 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.064+0000 7f7cb5e47640 1 -- 172.21.15.135:0/2124260791 wait complete. 2024-04-17T06:10:16.074 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:16.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.180+0000 7fc4f7724640 1 Processor -- start 2024-04-17T06:10:16.178 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.180+0000 7fc4f7724640 1 -- start start 2024-04-17T06:10:16.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f7724640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4f00cd0a0 0x7fc4f00cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f7724640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc4f015cc80 0x7fc4f015d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f7724640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f006ba30 con 0x7fc4f015c8a0 2024-04-17T06:10:16.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f7724640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f006c600 con 0x7fc4f015cc80 2024-04-17T06:10:16.179 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f7724640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f006bcd0 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f5cc1640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fc4f015cc80 0x7fc4f015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f54c0640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fc4f015c8a0 0x7fc4f00b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46356/0 (socket says 172.21.15.135:46356) 2024-04-17T06:10:16.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f54c0640 1 -- 172.21.15.135:0/3679242914 learned_addr learned my addr 172.21.15.135:0/3679242914 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:16.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f4cbf640 1 --2- 172.21.15.135:0/3679242914 >> v2:172.21.15.190:3300/0 conn(0x7fc4f00cd0a0 0x7fc4f00cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f5cc1640 1 -- 172.21.15.135:0/3679242914 >> v2:172.21.15.190:3300/0 conn(0x7fc4f00cd0a0 msgr2=0x7fc4f00cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f5cc1640 1 --2- 172.21.15.135:0/3679242914 >> v2:172.21.15.190:3300/0 conn(0x7fc4f00cd0a0 0x7fc4f00cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f5cc1640 1 -- 172.21.15.135:0/3679242914 >> v1:172.21.15.135:6789/0 conn(0x7fc4f015c8a0 legacy=0x7fc4f00b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.180 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.181+0000 7fc4f5cc1640 1 -- 172.21.15.135:0/3679242914 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f006bfb0 con 0x7fc4f015cc80 2024-04-17T06:10:16.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.182+0000 7fc4f5cc1640 1 --2- 172.21.15.135:0/3679242914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f015cc80 0x7fc4f015d110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00d2650 tx=0x7fc4e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa37d17ee48324a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.182+0000 7fc4e7fff640 1 -- 172.21.15.135:0/3679242914 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8039070 con 0x7fc4f015cc80 2024-04-17T06:10:16.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.182+0000 7fc4e7fff640 1 -- 172.21.15.135:0/3679242914 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e802cc80 con 0x7fc4f015cc80 2024-04-17T06:10:16.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.182+0000 7fc4e7fff640 1 -- 172.21.15.135:0/3679242914 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8034660 con 0x7fc4f015cc80 2024-04-17T06:10:16.181 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.183+0000 7fc4f7724640 1 -- 172.21.15.135:0/3679242914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f015cc80 msgr2=0x7fc4f015d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.183+0000 7fc4f7724640 1 --2- 172.21.15.135:0/3679242914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f015cc80 0x7fc4f015d110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00d2650 tx=0x7fc4e802bed0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.183+0000 7fc4f7724640 1 -- 172.21.15.135:0/3679242914 shutdown_connections 2024-04-17T06:10:16.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.183+0000 7fc4f7724640 1 --2- 172.21.15.135:0/3679242914 >> v2:172.21.15.190:3300/0 conn(0x7fc4f00cd0a0 0x7fc4f00cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.183+0000 7fc4f7724640 1 --2- 172.21.15.135:0/3679242914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f015cc80 0x7fc4f015d110 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.183+0000 7fc4f7724640 1 -- 172.21.15.135:0/3679242914 >> 172.21.15.135:0/3679242914 conn(0x7fc4f009a9f0 msgr2=0x7fc4f00b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.183+0000 7fc4f7724640 1 -- 172.21.15.135:0/3679242914 shutdown_connections 2024-04-17T06:10:16.182 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.183+0000 7fc4f7724640 1 -- 172.21.15.135:0/3679242914 wait complete. 2024-04-17T06:10:16.184 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.184+0000 7fc4f7724640 1 Processor -- start 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.184+0000 7fc4f7724640 1 -- start start 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f7724640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f00cd0a0 0x7fc4f00b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f7724640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4f015c8a0 0x7fc4f015b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f7724640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4f015cc80 0x7fc4f00c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f7724640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00ced60 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f7724640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00cda70 con 0x7fc4f015c8a0 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f7724640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f00b3f10 con 0x7fc4f015cc80 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f00cd0a0 0x7fc4f00b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f00cd0a0 0x7fc4f00b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42244/0 (socket says 172.21.15.135:42244) 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 -- 172.21.15.135:0/284934785 learned_addr learned my addr 172.21.15.135:0/284934785 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f5cc1640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4f015cc80 0x7fc4f00c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f4cbf640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4f015c8a0 0x7fc4f015b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 -- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4f015cc80 msgr2=0x7fc4f00c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.185 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4f015cc80 0x7fc4f00c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 -- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4f015c8a0 msgr2=0x7fc4f015b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4f015c8a0 0x7fc4f015b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 -- 172.21.15.135:0/284934785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00c8780 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f4cbf640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4f015c8a0 0x7fc4f015b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f5cc1640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4f015cc80 0x7fc4f00c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f54c0640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f00cd0a0 0x7fc4f00b41f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c400 tx=0x7fc4ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4e5ffb640 1 -- 172.21.15.135:0/284934785 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec017070 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4e5ffb640 1 -- 172.21.15.135:0/284934785 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec005ce0 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f00c8a60 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.185+0000 7fc4e5ffb640 1 -- 172.21.15.135:0/284934785 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec007310 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.186+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f00c62b0 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.186 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.187+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f0074940 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.187+0000 7fc4e5ffb640 1 -- 172.21.15.135:0/284934785 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4ec01a070 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.187+0000 7fc4e5ffb640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc4d406c8f0 0x7fc4d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.187+0000 7fc4e5ffb640 1 -- 172.21.15.135:0/284934785 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7fc4ec08b2a0 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.188+0000 7fc4f4cbf640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc4d406c8f0 0x7fc4d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.187 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.188+0000 7fc4f4cbf640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc4d406c8f0 0x7fc4d406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01f7860 tx=0x7fc4e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.189 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.192+0000 7fc4e5ffb640 1 -- 172.21.15.135:0/284934785 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4ec059360 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.367 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.369+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4f006df90 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.368 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.370+0000 7fc4e5ffb640 1 -- 172.21.15.135:0/284934785 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18950 (secure 0 0 0) 0x7fc4ec058c60 con 0x7fc4f00cd0a0 2024-04-17T06:10:16.368 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:16.369 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":26,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:01.008924+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:09:18.032781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-17T06:09:18.923217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-17T06:09:22.648961+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.1d","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:16.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc4d406c8f0 msgr2=0x7fc4d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.370 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc4d406c8f0 0x7fc4d406edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01f7860 tx=0x7fc4e0008040 comp rx=0 tx=0).stop 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f00cd0a0 msgr2=0x7fc4f00b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f00cd0a0 0x7fc4f00b41f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c400 tx=0x7fc4ec00c8c0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 shutdown_connections 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4f015cc80 0x7fc4f00c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4f015c8a0 0x7fc4f015b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fc4d406c8f0 0x7fc4d406edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 --2- 172.21.15.135:0/284934785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f00cd0a0 0x7fc4f00b41f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 >> 172.21.15.135:0/284934785 conn(0x7fc4f009a9f0 msgr2=0x7fc4f00b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 shutdown_connections 2024-04-17T06:10:16.371 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.372+0000 7fc4f7724640 1 -- 172.21.15.135:0/284934785 wait complete. 2024-04-17T06:10:16.382 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-17T06:10:16.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.488+0000 7f2926a35640 1 Processor -- start 2024-04-17T06:10:16.486 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.488+0000 7f2926a35640 1 -- start start 2024-04-17T06:10:16.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2926a35640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f292006b990 0x7f29200cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2926a35640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29200b5ae0 con 0x7f29200cfc40 2024-04-17T06:10:16.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2926a35640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29200b5dc0 con 0x7f292015ec70 2024-04-17T06:10:16.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2926a35640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f292006bdf0 con 0x7f292006b990 2024-04-17T06:10:16.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2917fff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f292006b990 0x7f29200cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2924fd2640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f29200cfc40 0x7f292015f050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46370/0 (socket says 172.21.15.135:46370) 2024-04-17T06:10:16.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2924fd2640 1 -- 172.21.15.135:0/1624624996 learned_addr learned my addr 172.21.15.135:0/1624624996 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:16.493 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2917fff640 1 -- 172.21.15.135:0/1624624996 >> v1:172.21.15.190:6789/0 conn(0x7f292015ec70 legacy=0x7f29200d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2917fff640 1 -- 172.21.15.135:0/1624624996 >> v1:172.21.15.135:6789/0 conn(0x7f29200cfc40 legacy=0x7f292015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f291f7fe640 1 -- 172.21.15.135:0/1624624996 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 557258094 0 0) 0x7f29200b5ae0 con 0x7f29200cfc40 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.489+0000 7f2917fff640 1 -- 172.21.15.135:0/1624624996 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29200b3e80 con 0x7f292006b990 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.490+0000 7f2917fff640 1 --2- 172.21.15.135:0/1624624996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29200cf700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f29080097a0 tx=0x7f290802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac0018e5c4eb88f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.490+0000 7f291f7fe640 1 -- 172.21.15.135:0/1624624996 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908035020 con 0x7f292006b990 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.490+0000 7f291f7fe640 1 -- 172.21.15.135:0/1624624996 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2908031830 con 0x7f292006b990 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.490+0000 7f291f7fe640 1 -- 172.21.15.135:0/1624624996 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908031c00 con 0x7f292006b990 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.491+0000 7f2926a35640 1 -- 172.21.15.135:0/1624624996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 msgr2=0x7f29200cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.491+0000 7f2926a35640 1 --2- 172.21.15.135:0/1624624996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29200cf700 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f29080097a0 tx=0x7f290802b8d0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.491+0000 7f2926a35640 1 -- 172.21.15.135:0/1624624996 shutdown_connections 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.491+0000 7f2926a35640 1 --2- 172.21.15.135:0/1624624996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29200cf700 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.494 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.491+0000 7f2926a35640 1 -- 172.21.15.135:0/1624624996 >> 172.21.15.135:0/1624624996 conn(0x7f29200cd7e0 msgr2=0x7f29200cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.491+0000 7f2926a35640 1 -- 172.21.15.135:0/1624624996 shutdown_connections 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.491+0000 7f2926a35640 1 -- 172.21.15.135:0/1624624996 wait complete. 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.492+0000 7f2926a35640 1 Processor -- start 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.492+0000 7f2926a35640 1 -- start start 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f2926a35640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29201e9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f2926a35640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29200cfc40 0x7f2920160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f2926a35640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f292015ec70 0x7f2920161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f2926a35640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29200d0070 con 0x7f292006b990 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f2926a35640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f29200b60a0 con 0x7f29200cfc40 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f2926a35640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f29200d01f0 con 0x7f292015ec70 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29201e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29201e9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42260/0 (socket says 172.21.15.135:42260) 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 -- 172.21.15.135:0/2608343809 learned_addr learned my addr 172.21.15.135:0/2608343809 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 -- 172.21.15.135:0/2608343809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f292015ec70 msgr2=0x7f2920161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.495 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f292015ec70 0x7f2920161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 -- 172.21.15.135:0/2608343809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29200cfc40 msgr2=0x7f2920160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29200cfc40 0x7f2920160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 -- 172.21.15.135:0/2608343809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292015f1f0 con 0x7f292006b990 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291ffff640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29201e9410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f291000c400 tx=0x7f291000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.493+0000 7f291d7fa640 1 -- 172.21.15.135:0/2608343809 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910017070 con 0x7f292006b990 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.494+0000 7f291d7fa640 1 -- 172.21.15.135:0/2608343809 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29100052d0 con 0x7f292006b990 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.494+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292015f4d0 con 0x7f292006b990 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.494+0000 7f291d7fa640 1 -- 172.21.15.135:0/2608343809 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910003830 con 0x7f292006b990 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.494+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292015fa90 con 0x7f292006b990 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.495+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2920074940 con 0x7f292006b990 2024-04-17T06:10:16.496 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.495+0000 7f291d7fa640 1 -- 172.21.15.135:0/2608343809 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f291001a070 con 0x7f292006b990 2024-04-17T06:10:16.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.495+0000 7f291d7fa640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.496+0000 7f291d7fa640 1 -- 172.21.15.135:0/2608343809 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f291008b2b0 con 0x7f292006b990 2024-04-17T06:10:16.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.496+0000 7f2917fff640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.497 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.496+0000 7f2917fff640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f290006c8f0 0x7f290006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2908036000 tx=0x7f2908030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.498 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.500+0000 7f291d7fa640 1 -- 172.21.15.135:0/2608343809 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29100592e0 con 0x7f292006b990 2024-04-17T06:10:16.714 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.716+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f29200751b0 con 0x7f292006b990 2024-04-17T06:10:16.714 INFO:tasks.ceph.mon.a.smithi135.stderr:2024-04-17T06:10:16.716+0000 7efd9ef85640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-17T06:10:16.715 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.717+0000 7f291d7fa640 1 -- 172.21.15.135:0/2608343809 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f2910058be0 con 0x7f292006b990 2024-04-17T06:10:16.715 INFO:teuthology.orchestra.run.smithi135.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f290006c8f0 msgr2=0x7f290006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f290006c8f0 0x7f290006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2908036000 tx=0x7f2908030000 comp rx=0 tx=0).stop 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 msgr2=0x7f29201e9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29201e9410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f291000c400 tx=0x7f291000c8c0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 shutdown_connections 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f292015ec70 0x7f2920161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29200cfc40 0x7f2920160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f290006c8f0 0x7f290006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.718 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 --2- 172.21.15.135:0/2608343809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f292006b990 0x7f29201e9410 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 >> 172.21.15.135:0/2608343809 conn(0x7f29200cd7e0 msgr2=0x7f29200b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.719+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 shutdown_connections 2024-04-17T06:10:16.719 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.720+0000 7f2926a35640 1 -- 172.21.15.135:0/2608343809 wait complete. 2024-04-17T06:10:16.723 INFO:tasks.ceph.mds.d.smithi190.stderr: -5251> 2024-04-17T06:09:24.584+0000 7f206f814640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T06:10:16.729 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-17T06:10:16.753 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2024-04-17T06:10:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.837+0000 7f3ee0be8640 1 Processor -- start 2024-04-17T06:10:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.837+0000 7f3ee0be8640 1 -- start start 2024-04-17T06:10:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3ee0be8640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3edc0b3cb0 0x7f3edc15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3ee0be8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc06ba30 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3ee0be8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc06bcd0 con 0x7f3edc0b3800 2024-04-17T06:10:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3ee0be8640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f3edc06c600 con 0x7f3edc0b3cb0 2024-04-17T06:10:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3eda59c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f3edc0cd1a0 0x7f3edc0cd580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46374/0 (socket says 172.21.15.135:46374) 2024-04-17T06:10:16.838 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3eda59c640 1 -- 172.21.15.135:0/886806531 learned_addr learned my addr 172.21.15.135:0/886806531 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3edad9d640 1 --2- 172.21.15.135:0/886806531 >> v2:172.21.15.190:3300/0 conn(0x7f3edc0b3cb0 0x7f3edc15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3ed9d9b640 1 -- 172.21.15.135:0/886806531 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2519608368 0 0) 0x7f3edc06ba30 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3ed9d9b640 1 -- 172.21.15.135:0/886806531 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ec8003520 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3ed9d9b640 1 -- 172.21.15.135:0/886806531 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540430183 0 0) 0x7f3edc06bcd0 con 0x7f3edc0b3800 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3ed9d9b640 1 -- 172.21.15.135:0/886806531 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3edc06ba30 con 0x7f3edc0b3800 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3edad9d640 1 -- 172.21.15.135:0/886806531 >> v1:172.21.15.190:6789/0 conn(0x7f3edc0b3800 legacy=0x7f3edc15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3edad9d640 1 -- 172.21.15.135:0/886806531 >> v1:172.21.15.135:6789/0 conn(0x7f3edc0cd1a0 legacy=0x7f3edc0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.838+0000 7f3edad9d640 1 -- 172.21.15.135:0/886806531 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc06bfb0 con 0x7f3edc0b3cb0 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.839+0000 7f3edad9d640 1 --2- 172.21.15.135:0/886806531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3cb0 0x7f3edc15d0b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009d90 tx=0x7f3ed402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14f744d897b5073d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.839+0000 7f3ed9d9b640 1 -- 172.21.15.135:0/886806531 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4039070 con 0x7f3edc0b3cb0 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.839+0000 7f3ed9d9b640 1 -- 172.21.15.135:0/886806531 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed402cc60 con 0x7f3edc0b3cb0 2024-04-17T06:10:16.839 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.839+0000 7f3ed9d9b640 1 -- 172.21.15.135:0/886806531 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4034640 con 0x7f3edc0b3cb0 2024-04-17T06:10:16.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.840+0000 7f3ee0be8640 1 -- 172.21.15.135:0/886806531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3cb0 msgr2=0x7f3edc15d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.840+0000 7f3ee0be8640 1 --2- 172.21.15.135:0/886806531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3cb0 0x7f3edc15d0b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009d90 tx=0x7f3ed402beb0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.840+0000 7f3ee0be8640 1 -- 172.21.15.135:0/886806531 shutdown_connections 2024-04-17T06:10:16.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.840+0000 7f3ee0be8640 1 --2- 172.21.15.135:0/886806531 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3cb0 0x7f3edc15d0b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.840+0000 7f3ee0be8640 1 -- 172.21.15.135:0/886806531 >> 172.21.15.135:0/886806531 conn(0x7f3edc09a9f0 msgr2=0x7f3edc0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:16.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.840+0000 7f3ee0be8640 1 -- 172.21.15.135:0/886806531 shutdown_connections 2024-04-17T06:10:16.840 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.840+0000 7f3ee0be8640 1 -- 172.21.15.135:0/886806531 wait complete. 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.841+0000 7f3ee0be8640 1 Processor -- start 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.841+0000 7f3ee0be8640 1 -- start start 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3ee0be8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3800 0x7f3edc0c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3ee0be8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3edc0b3cb0 0x7f3edc155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3ee0be8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3edc0cd1a0 0x7f3edc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3ee0be8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0d0370 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3ee0be8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0ced60 con 0x7f3edc0b3800 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3ee0be8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc0d04f0 con 0x7f3edc0b3cb0 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3edad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3edc0cd1a0 0x7f3edc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3edad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3edc0cd1a0 0x7f3edc15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42276/0 (socket says 172.21.15.135:42276) 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3edad9d640 1 -- 172.21.15.135:0/2773856250 learned_addr learned my addr 172.21.15.135:0/2773856250 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3eda59c640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3800 0x7f3edc0c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3ed1d9b640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3edc0b3cb0 0x7f3edc155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3edad9d640 1 -- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3edc0b3cb0 msgr2=0x7f3edc155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3edad9d640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3edc0b3cb0 0x7f3edc155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3edad9d640 1 -- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3800 msgr2=0x7f3edc0c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3edad9d640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3800 0x7f3edc0c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3edad9d640 1 -- 172.21.15.135:0/2773856250 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc0c7aa0 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.842+0000 7f3eda59c640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3800 0x7f3edc0c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.843+0000 7f3edad9d640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3edc0cd1a0 0x7f3edc15ecb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009d90 tx=0x7f3ed4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.843+0000 7f3ed37fe640 1 -- 172.21.15.135:0/2773856250 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4039050 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.843+0000 7f3ed37fe640 1 -- 172.21.15.135:0/2773856250 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed4045070 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.843+0000 7f3ed37fe640 1 -- 172.21.15.135:0/2773856250 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4009450 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.843+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc0c7d80 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.843+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc0b5620 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.844+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc074940 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.844+0000 7f3ed37fe640 1 -- 172.21.15.135:0/2773856250 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ed403e830 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.845+0000 7f3ed37fe640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.845+0000 7f3eda59c640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.845+0000 7f3ed37fe640 1 -- 172.21.15.135:0/2773856250 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7f3ed40aec40 con 0x7f3edc0cd1a0 2024-04-17T06:10:16.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.845+0000 7f3eda59c640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000f60 tx=0x7f3ecc009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:16.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:16.849+0000 7f3ed37fe640 1 -- 172.21.15.135:0/2773856250 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ed4032e20 con 0x7f3edc0cd1a0 2024-04-17T06:10:17.061 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.063+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3edc0af710 con 0x7f3edc0cd1a0 2024-04-17T06:10:17.717 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.719+0000 7f3ed37fe640 1 -- 172.21.15.135:0/2773856250 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f3ed407cd70 con 0x7f3edc0cd1a0 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.721+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3ebc06c8f0 msgr2=0x7f3ebc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.721+0000 7f3ee0be8640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc000f60 tx=0x7f3ecc009040 comp rx=0 tx=0).stop 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.721+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3edc0cd1a0 msgr2=0x7f3edc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.721+0000 7f3ee0be8640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3edc0cd1a0 0x7f3edc15ecb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009d90 tx=0x7f3ed4004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.722+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 shutdown_connections 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.722+0000 7f3ee0be8640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f3ebc06c8f0 0x7f3ebc06edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.722+0000 7f3ee0be8640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3edc0cd1a0 0x7f3edc15ecb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.722+0000 7f3ee0be8640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3edc0b3cb0 0x7f3edc155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.722+0000 7f3ee0be8640 1 --2- 172.21.15.135:0/2773856250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3edc0b3800 0x7f3edc0c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.722+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 >> 172.21.15.135:0/2773856250 conn(0x7f3edc09a9f0 msgr2=0x7f3edc0ca660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:17.720 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.722+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 shutdown_connections 2024-04-17T06:10:17.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.722+0000 7f3ee0be8640 1 -- 172.21.15.135:0/2773856250 wait complete. 2024-04-17T06:10:17.732 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-17T06:10:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.844+0000 7fb8c8152640 1 Processor -- start 2024-04-17T06:10:17.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.844+0000 7fb8c8152640 1 -- start start 2024-04-17T06:10:17.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c8152640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c015eca0 0x7fb8c00d4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:17.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c8152640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8c00d1a60 0x7fb8c00b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:17.843 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c8152640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c00d33d0 con 0x7fb8c00d29f0 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c8152640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c015f080 con 0x7fb8c00d1a60 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c8152640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c00d36b0 con 0x7fb8c015eca0 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c66ef640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb8c00d1a60 0x7fb8c00b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c56ed640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb8c00d29f0 0x7fb8c00d1350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46384/0 (socket says 172.21.15.135:46384) 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c56ed640 1 -- 172.21.15.135:0/1380918059 learned_addr learned my addr 172.21.15.135:0/1380918059 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c5eee640 1 --2- 172.21.15.135:0/1380918059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c015eca0 0x7fb8c00d4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c66ef640 1 -- 172.21.15.135:0/1380918059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c015eca0 msgr2=0x7fb8c00d4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c66ef640 1 --2- 172.21.15.135:0/1380918059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c015eca0 0x7fb8c00d4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c66ef640 1 -- 172.21.15.135:0/1380918059 >> v1:172.21.15.135:6789/0 conn(0x7fb8c00d29f0 legacy=0x7fb8c00d1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c4eec640 1 -- 172.21.15.135:0/1380918059 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1924138812 0 0) 0x7fb8c00d33d0 con 0x7fb8c00d29f0 2024-04-17T06:10:17.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c66ef640 1 -- 172.21.15.135:0/1380918059 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c00d3990 con 0x7fb8c00d1a60 2024-04-17T06:10:17.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.845+0000 7fb8c5eee640 1 --2- 172.21.15.135:0/1380918059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c015eca0 0x7fb8c00d4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.846+0000 7fb8c66ef640 1 --2- 172.21.15.135:0/1380918059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d1a60 0x7fb8c00b7ae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009d90 tx=0x7fb8b802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7513522ebb3a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.846+0000 7fb8c4eec640 1 -- 172.21.15.135:0/1380918059 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8039070 con 0x7fb8c00d1a60 2024-04-17T06:10:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.846+0000 7fb8c4eec640 1 -- 172.21.15.135:0/1380918059 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b802cce0 con 0x7fb8c00d1a60 2024-04-17T06:10:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.846+0000 7fb8c4eec640 1 -- 172.21.15.135:0/1380918059 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8034660 con 0x7fb8c00d1a60 2024-04-17T06:10:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.847+0000 7fb8c8152640 1 -- 172.21.15.135:0/1380918059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d1a60 msgr2=0x7fb8c00b7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:17.846 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.847+0000 7fb8c8152640 1 --2- 172.21.15.135:0/1380918059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d1a60 0x7fb8c00b7ae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009d90 tx=0x7fb8b802bf30 comp rx=0 tx=0).stop 2024-04-17T06:10:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.847+0000 7fb8c8152640 1 -- 172.21.15.135:0/1380918059 shutdown_connections 2024-04-17T06:10:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.847+0000 7fb8c8152640 1 --2- 172.21.15.135:0/1380918059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d1a60 0x7fb8c00b7ae0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.847+0000 7fb8c8152640 1 --2- 172.21.15.135:0/1380918059 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c015eca0 0x7fb8c00d4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.847+0000 7fb8c8152640 1 -- 172.21.15.135:0/1380918059 >> 172.21.15.135:0/1380918059 conn(0x7fb8c00b64e0 msgr2=0x7fb8c00b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.847+0000 7fb8c8152640 1 -- 172.21.15.135:0/1380918059 shutdown_connections 2024-04-17T06:10:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.847+0000 7fb8c8152640 1 -- 172.21.15.135:0/1380918059 wait complete. 2024-04-17T06:10:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.848+0000 7fb8c8152640 1 Processor -- start 2024-04-17T06:10:17.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.848+0000 7fb8c8152640 1 -- start start 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c8152640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c00d1a60 0x7fb8c01567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c8152640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d29f0 0x7fb8c01f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c8152640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8c015eca0 0x7fb8c01f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c8152640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c00ba490 con 0x7fb8c00d29f0 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c8152640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c00d3f50 con 0x7fb8c015eca0 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c8152640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c00ba610 con 0x7fb8c00d1a60 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d29f0 0x7fb8c01f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d29f0 0x7fb8c01f4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42296/0 (socket says 172.21.15.135:42296) 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 -- 172.21.15.135:0/847174444 learned_addr learned my addr 172.21.15.135:0/847174444 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:17.850 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c5eee640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c00d1a60 0x7fb8c01567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c66ef640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8c015eca0 0x7fb8c01f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 -- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c00d1a60 msgr2=0x7fb8c01567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c00d1a60 0x7fb8c01567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 -- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8c015eca0 msgr2=0x7fb8c01f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8c015eca0 0x7fb8c01f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 -- 172.21.15.135:0/847174444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c00caf10 con 0x7fb8c00d29f0 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c66ef640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8c015eca0 0x7fb8c01f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8c56ed640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d29f0 0x7fb8c01f4130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c4d0 tx=0x7fb8b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8b67fc640 1 -- 172.21.15.135:0/847174444 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0018070 con 0x7fb8c00d29f0 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8b67fc640 1 -- 172.21.15.135:0/847174444 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b0007500 con 0x7fb8c00d29f0 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.849+0000 7fb8b67fc640 1 -- 172.21.15.135:0/847174444 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0004a20 con 0x7fb8c00d29f0 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.850+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c00b87e0 con 0x7fb8c00d29f0 2024-04-17T06:10:17.851 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.850+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c00b8da0 con 0x7fb8c00d29f0 2024-04-17T06:10:17.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.851+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c00749b0 con 0x7fb8c00d29f0 2024-04-17T06:10:17.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.851+0000 7fb8b67fc640 1 -- 172.21.15.135:0/847174444 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8b0004bc0 con 0x7fb8c00d29f0 2024-04-17T06:10:17.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.852+0000 7fb8b67fc640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8a406c920 0x7fb8a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:17.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.852+0000 7fb8c5eee640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8a406c920 0x7fb8a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:17.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.852+0000 7fb8b67fc640 1 -- 172.21.15.135:0/847174444 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6821+0+0 (secure 0 0 0) 0x7fb8b008b1d0 con 0x7fb8c00d29f0 2024-04-17T06:10:17.852 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.852+0000 7fb8c5eee640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8a406c920 0x7fb8a406ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00aac0 tx=0x7fb8bc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:17.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:17.856+0000 7fb8b67fc640 1 -- 172.21.15.135:0/847174444 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8b0058b50 con 0x7fb8c00d29f0 2024-04-17T06:10:18.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.035+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb8c00b9ea0 con 0x7fb8c00d29f0 2024-04-17T06:10:18.721 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.723+0000 7fb8b67fc640 1 -- 172.21.15.135:0/847174444 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) ==== 181+0+0 (secure 0 0 0) 0x7fb8b005ce50 con 0x7fb8c00d29f0 2024-04-17T06:10:18.721 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'cephfs_metadata' removed 2024-04-17T06:10:18.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.726+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8a406c920 msgr2=0x7fb8a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:18.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.726+0000 7fb8c8152640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8a406c920 0x7fb8a406ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00aac0 tx=0x7fb8bc008040 comp rx=0 tx=0).stop 2024-04-17T06:10:18.724 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.726+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d29f0 msgr2=0x7fb8c01f4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:18.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.726+0000 7fb8c8152640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d29f0 0x7fb8c01f4130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000c4d0 tx=0x7fb8b000c990 comp rx=0 tx=0).stop 2024-04-17T06:10:18.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.727+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 shutdown_connections 2024-04-17T06:10:18.725 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.727+0000 7fb8c8152640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb8c015eca0 0x7fb8c01f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.727+0000 7fb8c8152640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb8a406c920 0x7fb8a406ede0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.727+0000 7fb8c8152640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8c00d29f0 0x7fb8c01f4130 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.727+0000 7fb8c8152640 1 --2- 172.21.15.135:0/847174444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb8c00d1a60 0x7fb8c01567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.727+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 >> 172.21.15.135:0/847174444 conn(0x7fb8c00b64e0 msgr2=0x7fb8c00b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:18.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.727+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 shutdown_connections 2024-04-17T06:10:18.726 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.727+0000 7fb8c8152640 1 -- 172.21.15.135:0/847174444 wait complete. 2024-04-17T06:10:18.737 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-17T06:10:18.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.848+0000 7f389024a640 1 Processor -- start 2024-04-17T06:10:18.847 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.849+0000 7f389024a640 1 -- start start 2024-04-17T06:10:18.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.849+0000 7f389024a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f388809b1e0 0x7f38880b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:18.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.849+0000 7f389024a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f38880b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:18.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.849+0000 7f389024a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38880d0750 con 0x7f3888160290 2024-04-17T06:10:18.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.849+0000 7f389024a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f38880cd2a0 con 0x7f388809b1e0 2024-04-17T06:10:18.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.849+0000 7f389024a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f38880d10a0 con 0x7f38880ce720 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388d7e5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f388809b1e0 0x7f38880b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388e7e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f38880b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388dfe6640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f3888160290 0x7f388809aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:40250/0 (socket says 172.21.15.135:40250) 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388dfe6640 1 -- 172.21.15.135:0/2819718786 learned_addr learned my addr 172.21.15.135:0/2819718786 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388d7e5640 1 -- 172.21.15.135:0/2819718786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 msgr2=0x7f38880b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388d7e5640 1 --2- 172.21.15.135:0/2819718786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f38880b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388d7e5640 1 -- 172.21.15.135:0/2819718786 >> v1:172.21.15.190:6789/0 conn(0x7f3888160290 legacy=0x7f388809aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388d7e5640 1 -- 172.21.15.135:0/2819718786 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38880d1380 con 0x7f388809b1e0 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.850+0000 7f388e7e7640 1 --2- 172.21.15.135:0/2819718786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f38880b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.851+0000 7f388d7e5640 1 --2- 172.21.15.135:0/2819718786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 0x7f38880b1d60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f387c0097a0 tx=0x7f387c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf12b66dcec8ead server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.851+0000 7f388cfe4640 1 -- 172.21.15.135:0/2819718786 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0315a0 con 0x7f388809b1e0 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.851+0000 7f388cfe4640 1 -- 172.21.15.135:0/2819718786 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387c037070 con 0x7f388809b1e0 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.851+0000 7f388cfe4640 1 -- 172.21.15.135:0/2819718786 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c031960 con 0x7f388809b1e0 2024-04-17T06:10:18.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.852+0000 7f389024a640 1 -- 172.21.15.135:0/2819718786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 msgr2=0x7f38880b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.852+0000 7f389024a640 1 --2- 172.21.15.135:0/2819718786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 0x7f38880b1d60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f387c0097a0 tx=0x7f387c02b840 comp rx=0 tx=0).stop 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.852+0000 7f389024a640 1 -- 172.21.15.135:0/2819718786 shutdown_connections 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.852+0000 7f389024a640 1 --2- 172.21.15.135:0/2819718786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f38880b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.852+0000 7f389024a640 1 --2- 172.21.15.135:0/2819718786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 0x7f38880b1d60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.852+0000 7f389024a640 1 -- 172.21.15.135:0/2819718786 >> 172.21.15.135:0/2819718786 conn(0x7f38880b4460 msgr2=0x7f38880b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.852+0000 7f389024a640 1 -- 172.21.15.135:0/2819718786 shutdown_connections 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.852+0000 7f389024a640 1 -- 172.21.15.135:0/2819718786 wait complete. 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.853+0000 7f389024a640 1 Processor -- start 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.853+0000 7f389024a640 1 -- start start 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f389024a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 0x7f3888152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f389024a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f3888157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f389024a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3888160290 0x7f38880b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f389024a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f388809b7f0 con 0x7f3888160290 2024-04-17T06:10:18.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f389024a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f388809b670 con 0x7f388809b1e0 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f389024a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f388809b970 con 0x7f38880ce720 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3888160290 0x7f38880b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3888160290 0x7f38880b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42312/0 (socket says 172.21.15.135:42312) 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 -- 172.21.15.135:0/2660441744 learned_addr learned my addr 172.21.15.135:0/2660441744 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388d7e5640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f3888157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388dfe6640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 0x7f3888152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 -- 172.21.15.135:0/2660441744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 msgr2=0x7f3888157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f3888157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 -- 172.21.15.135:0/2660441744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 msgr2=0x7f3888152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 0x7f3888152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 -- 172.21.15.135:0/2660441744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38881532e0 con 0x7f3888160290 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.854+0000 7f388e7e7640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3888160290 0x7f38880b5590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f388000e850 tx=0x7f388000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:18.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.855+0000 7f38867fc640 1 -- 172.21.15.135:0/2660441744 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880017070 con 0x7f3888160290 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.855+0000 7f38867fc640 1 -- 172.21.15.135:0/2660441744 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388000c940 con 0x7f3888160290 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.855+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38880b6100 con 0x7f3888160290 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.855+0000 7f38867fc640 1 -- 172.21.15.135:0/2660441744 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388000ccf0 con 0x7f3888160290 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.855+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38880b66c0 con 0x7f3888160290 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.856+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3888068a20 con 0x7f3888160290 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.856+0000 7f38867fc640 1 -- 172.21.15.135:0/2660441744 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f388001a070 con 0x7f3888160290 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.857+0000 7f38867fc640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f387406c8f0 0x7f387406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.857+0000 7f388dfe6640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f387406c8f0 0x7f387406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.857+0000 7f38867fc640 1 -- 172.21.15.135:0/2660441744 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6383+0+0 (secure 0 0 0) 0x7f388008bb90 con 0x7f3888160290 2024-04-17T06:10:18.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.857+0000 7f388dfe6640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f387406c8f0 0x7f387406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f38780012e0 tx=0x7f3878001990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:18.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:18.861+0000 7f38867fc640 1 -- 172.21.15.135:0/2660441744 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f38800596d0 con 0x7f3888160290 2024-04-17T06:10:19.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.044+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f388815f600 con 0x7f3888160290 2024-04-17T06:10:19.528 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:10:19.531 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:10:19.728 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.729+0000 7f38867fc640 1 -- 172.21.15.135:0/2660441744 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) ==== 169+0+0 (secure 0 0 0) 0x7f388005d9d0 con 0x7f3888160290 2024-04-17T06:10:19.728 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'cephfs_data' removed 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f387406c8f0 msgr2=0x7f387406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f387406c8f0 0x7f387406edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f38780012e0 tx=0x7f3878001990 comp rx=0 tx=0).stop 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3888160290 msgr2=0x7f38880b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3888160290 0x7f38880b5590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f388000e850 tx=0x7f388000ed10 comp rx=0 tx=0).stop 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 shutdown_connections 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f387406c8f0 0x7f387406edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3888160290 0x7f38880b5590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38880ce720 0x7f3888157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.731 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 --2- 172.21.15.135:0/2660441744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f388809b1e0 0x7f3888152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 >> 172.21.15.135:0/2660441744 conn(0x7f38880b4460 msgr2=0x7f38880cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:19.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 shutdown_connections 2024-04-17T06:10:19.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.733+0000 7f389024a640 1 -- 172.21.15.135:0/2660441744 wait complete. 2024-04-17T06:10:19.744 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-17T06:10:19.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.857+0000 7f46fd182640 1 Processor -- start 2024-04-17T06:10:19.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.857+0000 7f46fd182640 1 -- start start 2024-04-17T06:10:19.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46fd182640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46f81457f0 0x7f46f80b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:19.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46fd182640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46f80b1f90 0x7f46f80d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:19.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46fd182640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f80b6110 con 0x7f46f8145ce0 2024-04-17T06:10:19.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46fd182640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f46f80b6d20 con 0x7f46f81457f0 2024-04-17T06:10:19.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46fd182640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f46f80b63f0 con 0x7f46f80b1f90 2024-04-17T06:10:19.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46f6d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46f81457f0 0x7f46f80b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:19.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46f6d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f46f81457f0 0x7f46f80b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42322/0 (socket says 172.21.15.135:42322) 2024-04-17T06:10:19.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46f6d9d640 1 -- 172.21.15.135:0/1175172470 learned_addr learned my addr 172.21.15.135:0/1175172470 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:19.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46effff640 1 --2- 172.21.15.135:0/1175172470 >> v2:172.21.15.190:3300/0 conn(0x7f46f80b1f90 0x7f46f80d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:19.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46f6d9d640 1 -- 172.21.15.135:0/1175172470 >> v2:172.21.15.190:3300/0 conn(0x7f46f80b1f90 msgr2=0x7f46f80d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:19.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46f6d9d640 1 --2- 172.21.15.135:0/1175172470 >> v2:172.21.15.190:3300/0 conn(0x7f46f80b1f90 0x7f46f80d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.858+0000 7f46f6d9d640 1 -- 172.21.15.135:0/1175172470 >> v1:172.21.15.190:6789/0 conn(0x7f46f8145ce0 legacy=0x7f46f815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:19.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.859+0000 7f46f6d9d640 1 -- 172.21.15.135:0/1175172470 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f80b66d0 con 0x7f46f81457f0 2024-04-17T06:10:19.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46f6d9d640 1 --2- 172.21.15.135:0/1175172470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f81457f0 0x7f46f80b71b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f46f80b6ea0 tx=0x7f46e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390107c10144be8f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:19.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46f659c640 1 -- 172.21.15.135:0/1175172470 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8034070 con 0x7f46f81457f0 2024-04-17T06:10:19.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46f659c640 1 -- 172.21.15.135:0/1175172470 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e8030860 con 0x7f46f81457f0 2024-04-17T06:10:19.859 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46f659c640 1 -- 172.21.15.135:0/1175172470 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8030c30 con 0x7f46f81457f0 2024-04-17T06:10:19.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46fd182640 1 -- 172.21.15.135:0/1175172470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f81457f0 msgr2=0x7f46f80b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:19.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46fd182640 1 --2- 172.21.15.135:0/1175172470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f81457f0 0x7f46f80b71b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f46f80b6ea0 tx=0x7f46e802b9a0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46fd182640 1 -- 172.21.15.135:0/1175172470 shutdown_connections 2024-04-17T06:10:19.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46fd182640 1 --2- 172.21.15.135:0/1175172470 >> v2:172.21.15.190:3300/0 conn(0x7f46f80b1f90 0x7f46f80d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46fd182640 1 --2- 172.21.15.135:0/1175172470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f81457f0 0x7f46f80b71b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.860+0000 7f46fd182640 1 -- 172.21.15.135:0/1175172470 >> 172.21.15.135:0/1175172470 conn(0x7f46f80b4400 msgr2=0x7f46f80b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:19.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.861+0000 7f46fd182640 1 -- 172.21.15.135:0/1175172470 shutdown_connections 2024-04-17T06:10:19.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.861+0000 7f46fd182640 1 -- 172.21.15.135:0/1175172470 wait complete. 2024-04-17T06:10:19.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.862+0000 7f46fd182640 1 Processor -- start 2024-04-17T06:10:19.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.862+0000 7f46fd182640 1 -- start start 2024-04-17T06:10:19.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46fd182640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80b1f90 0x7f46f81d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:19.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46fd182640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f81457f0 0x7f46f8143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:19.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46fd182640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f8145ce0 0x7f46f80aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:19.862 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46fd182640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46f81463b0 con 0x7f46f8145ce0 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46fd182640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f46f8146230 con 0x7f46f80b1f90 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46fd182640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f46f8146530 con 0x7f46f81457f0 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f8145ce0 0x7f46f80aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f8145ce0 0x7f46f80aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42330/0 (socket says 172.21.15.135:42330) 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 -- 172.21.15.135:0/586327031 learned_addr learned my addr 172.21.15.135:0/586327031 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 -- 172.21.15.135:0/586327031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f81457f0 msgr2=0x7f46f8143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f81457f0 0x7f46f8143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 -- 172.21.15.135:0/586327031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80b1f90 msgr2=0x7f46f81d32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80b1f90 0x7f46f81d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 -- 172.21.15.135:0/586327031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f80ab360 con 0x7f46f8145ce0 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.863+0000 7f46f759e640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f8145ce0 0x7f46f80aa4a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c730 tx=0x7f46f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.864+0000 7f46ef7fe640 1 -- 172.21.15.135:0/586327031 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0003b70 con 0x7f46f8145ce0 2024-04-17T06:10:19.863 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.864+0000 7f46ef7fe640 1 -- 172.21.15.135:0/586327031 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f0003d10 con 0x7f46f8145ce0 2024-04-17T06:10:19.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.864+0000 7f46ef7fe640 1 -- 172.21.15.135:0/586327031 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f000aa90 con 0x7f46f8145ce0 2024-04-17T06:10:19.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.864+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f80ab080 con 0x7f46f8145ce0 2024-04-17T06:10:19.864 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.864+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f81d2ab0 con 0x7f46f8145ce0 2024-04-17T06:10:19.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.865+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f8068a20 con 0x7f46f8145ce0 2024-04-17T06:10:19.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.865+0000 7f46ef7fe640 1 -- 172.21.15.135:0/586327031 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46f000ac30 con 0x7f46f8145ce0 2024-04-17T06:10:19.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.866+0000 7f46ef7fe640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f46d806c8f0 0x7f46d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:19.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.866+0000 7f46ef7fe640 1 -- 172.21.15.135:0/586327031 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5853+0+0 (secure 0 0 0) 0x7f46f008bad0 con 0x7f46f8145ce0 2024-04-17T06:10:19.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.866+0000 7f46f6d9d640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f46d806c8f0 0x7f46d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:19.865 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.866+0000 7f46f6d9d640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f46d806c8f0 0x7f46d806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46e8036000 tx=0x7f46e803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:19.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:19.870+0000 7f46ef7fe640 1 -- 172.21.15.135:0/586327031 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f46f0059e70 con 0x7f46f8145ce0 2024-04-17T06:10:20.052 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.054+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46f81e8510 con 0x7f46f8145ce0 2024-04-17T06:10:20.732 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.734+0000 7f46ef7fe640 1 -- 172.21.15.135:0/586327031 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) ==== 178+0+0 (secure 0 0 0) 0x7f46f0059770 con 0x7f46f8145ce0 2024-04-17T06:10:20.733 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'cephfs_data_ec' removed 2024-04-17T06:10:20.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.738+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f46d806c8f0 msgr2=0x7f46d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:20.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.738+0000 7f46fd182640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f46d806c8f0 0x7f46d806edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46e8036000 tx=0x7f46e803b040 comp rx=0 tx=0).stop 2024-04-17T06:10:20.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.738+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f8145ce0 msgr2=0x7f46f80aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:20.737 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.738+0000 7f46fd182640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f8145ce0 0x7f46f80aa4a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c730 tx=0x7f46f000cbf0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.739+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 shutdown_connections 2024-04-17T06:10:20.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.739+0000 7f46fd182640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f46d806c8f0 0x7f46d806edb0 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46e8036000 tx=0x7f46e803b040 comp rx=0 tx=0).stop 2024-04-17T06:10:20.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.739+0000 7f46fd182640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f46f8145ce0 0x7f46f80aa4a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.739+0000 7f46fd182640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f81457f0 0x7f46f8143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.739+0000 7f46fd182640 1 --2- 172.21.15.135:0/586327031 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80b1f90 0x7f46f81d32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.738 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.739+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 >> 172.21.15.135:0/586327031 conn(0x7f46f80b4400 msgr2=0x7f46f80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:20.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.739+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 shutdown_connections 2024-04-17T06:10:20.739 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.740+0000 7f46fd182640 1 -- 172.21.15.135:0/586327031 wait complete. 2024-04-17T06:10:20.753 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-17T06:10:20.753 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-17T06:10:20.754 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-17T06:10:20.754 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T06:10:20.754 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-17T06:10:20.754 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-17T06:10:20.755 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-17T06:10:20.755 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T06:10:20.755 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-17T06:10:20.755 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-17T06:10:20.756 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-17T06:10:20.756 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T06:10:20.756 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-17T06:10:20.757 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-17T06:10:20.757 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-17T06:10:20.757 INFO:teuthology.orchestra.run:waiting for 300 2024-04-17T06:10:20.764 INFO:tasks.ceph.mds.a:Stopped 2024-04-17T06:10:20.765 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-17T06:10:20.766 INFO:tasks.ceph.mds.c:Stopped 2024-04-17T06:10:20.769 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-17T06:10:20.771 INFO:tasks.ceph.mds.b:Stopped 2024-04-17T06:10:20.771 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-17T06:10:20.772 INFO:tasks.ceph.mds.d:Stopped 2024-04-17T06:10:20.773 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-17T06:10:20.807 INFO:tasks.ceph.mds.a:Started 2024-04-17T06:10:20.808 DEBUG:teuthology.parallel:result is None 2024-04-17T06:10:20.808 INFO:tasks.ceph.mds.c:Started 2024-04-17T06:10:20.808 DEBUG:teuthology.parallel:result is None 2024-04-17T06:10:20.813 INFO:tasks.ceph.mds.b:Started 2024-04-17T06:10:20.813 DEBUG:teuthology.parallel:result is None 2024-04-17T06:10:20.814 INFO:tasks.ceph.mds.d:Started 2024-04-17T06:10:20.814 DEBUG:teuthology.parallel:result is None 2024-04-17T06:10:20.815 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-17T06:10:20.888 INFO:tasks.ceph.mds.a.smithi135.stdout:starting mds.a at 2024-04-17T06:10:20.889 INFO:tasks.ceph.mds.c.smithi135.stdout:starting mds.c at 2024-04-17T06:10:20.899 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2024-04-17T06:10:20.900 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2024-04-17T06:10:20.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.925+0000 7f606f5db640 1 Processor -- start 2024-04-17T06:10:20.923 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.925+0000 7f606f5db640 1 -- start start 2024-04-17T06:10:20.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606f5db640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f606809a9f0 0x7f606809add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:20.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606f5db640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f60680c78d0 0x7f60680cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:20.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606f5db640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f606809b310 con 0x7f606815cc60 2024-04-17T06:10:20.930 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606f5db640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f60680d0c90 con 0x7f60680c78d0 2024-04-17T06:10:20.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606f5db640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f60680d0f70 con 0x7f606809a9f0 2024-04-17T06:10:20.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606db78640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f60680c78d0 0x7f60680cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:20.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606db78640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f60680c78d0 0x7f60680cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42350/0 (socket says 172.21.15.135:42350) 2024-04-17T06:10:20.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606db78640 1 -- 172.21.15.135:0/2434442090 learned_addr learned my addr 172.21.15.135:0/2434442090 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:20.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.926+0000 7f606d377640 1 --2- 172.21.15.135:0/2434442090 >> v2:172.21.15.190:3300/0 conn(0x7f606809a9f0 0x7f606809add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:20.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.927+0000 7f606db78640 1 -- 172.21.15.135:0/2434442090 >> v2:172.21.15.190:3300/0 conn(0x7f606809a9f0 msgr2=0x7f606809add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:20.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.927+0000 7f606db78640 1 --2- 172.21.15.135:0/2434442090 >> v2:172.21.15.190:3300/0 conn(0x7f606809a9f0 0x7f606809add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.931 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.927+0000 7f606db78640 1 -- 172.21.15.135:0/2434442090 >> v1:172.21.15.190:6789/0 conn(0x7f606815cc60 legacy=0x7f60680cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.927+0000 7f606db78640 1 -- 172.21.15.135:0/2434442090 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60680d1250 con 0x7f60680c78d0 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.927+0000 7f606db78640 1 --2- 172.21.15.135:0/2434442090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60680c78d0 0x7f60680cda80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f606806ff70 tx=0x7f606002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2490d1fe2d52563a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f605ffff640 1 -- 172.21.15.135:0/2434442090 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606002cb20 con 0x7f60680c78d0 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f605ffff640 1 -- 172.21.15.135:0/2434442090 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606002ccc0 con 0x7f60680c78d0 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f605ffff640 1 -- 172.21.15.135:0/2434442090 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060034610 con 0x7f60680c78d0 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f606f5db640 1 -- 172.21.15.135:0/2434442090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60680c78d0 msgr2=0x7f60680cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f606f5db640 1 --2- 172.21.15.135:0/2434442090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60680c78d0 0x7f60680cda80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f606806ff70 tx=0x7f606002bf80 comp rx=0 tx=0).stop 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f606f5db640 1 -- 172.21.15.135:0/2434442090 shutdown_connections 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f606f5db640 1 --2- 172.21.15.135:0/2434442090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60680c78d0 0x7f60680cda80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f606f5db640 1 --2- 172.21.15.135:0/2434442090 >> v2:172.21.15.190:3300/0 conn(0x7f606809a9f0 0x7f606809add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f606f5db640 1 -- 172.21.15.135:0/2434442090 >> 172.21.15.135:0/2434442090 conn(0x7f60680b41a0 msgr2=0x7f60680b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:20.932 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f606f5db640 1 -- 172.21.15.135:0/2434442090 shutdown_connections 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.928+0000 7f606f5db640 1 -- 172.21.15.135:0/2434442090 wait complete. 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.929+0000 7f606f5db640 1 Processor -- start 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606f5db640 1 -- start start 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606f5db640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f606809a9f0 0x7f60680b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606f5db640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60680c78d0 0x7f6068153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606f5db640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f606815cc60 0x7f606809fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606f5db640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60680cca60 con 0x7f606815cc60 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606f5db640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f60680c63a0 con 0x7f606809a9f0 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606f5db640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f606809bdf0 con 0x7f60680c78d0 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606db78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f606815cc60 0x7f606809fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:20.933 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606db78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f606815cc60 0x7f606809fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42352/0 (socket says 172.21.15.135:42352) 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.930+0000 7f606db78640 1 -- 172.21.15.135:0/3200529790 learned_addr learned my addr 172.21.15.135:0/3200529790 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606d377640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f606809a9f0 0x7f60680b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606cb76640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60680c78d0 0x7f6068153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606db78640 1 -- 172.21.15.135:0/3200529790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60680c78d0 msgr2=0x7f6068153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606db78640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60680c78d0 0x7f6068153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606db78640 1 -- 172.21.15.135:0/3200529790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f606809a9f0 msgr2=0x7f60680b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606db78640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f606809a9f0 0x7f60680b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606db78640 1 -- 172.21.15.135:0/3200529790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60680b7930 con 0x7f606815cc60 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606db78640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f606815cc60 0x7f606809fd50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6060009f40 tx=0x7f6060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:20.934 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f605dffb640 1 -- 172.21.15.135:0/3200529790 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060043020 con 0x7f606815cc60 2024-04-17T06:10:20.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f605dffb640 1 -- 172.21.15.135:0/3200529790 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606003e450 con 0x7f606815cc60 2024-04-17T06:10:20.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f605dffb640 1 -- 172.21.15.135:0/3200529790 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060009510 con 0x7f606815cc60 2024-04-17T06:10:20.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60680b7c10 con 0x7f606815cc60 2024-04-17T06:10:20.935 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.931+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60680a0290 con 0x7f606815cc60 2024-04-17T06:10:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.933+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60681f23e0 con 0x7f606815cc60 2024-04-17T06:10:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.933+0000 7f605dffb640 1 -- 172.21.15.135:0/3200529790 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6060046020 con 0x7f606815cc60 2024-04-17T06:10:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.933+0000 7f605dffb640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f604c06c8f0 0x7f604c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.933+0000 7f605dffb640 1 -- 172.21.15.135:0/3200529790 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5335+0+0 (secure 0 0 0) 0x7f60600ae2a0 con 0x7f606815cc60 2024-04-17T06:10:20.936 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.933+0000 7f606d377640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f604c06c8f0 0x7f604c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:20.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.934+0000 7f606d377640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f604c06c8f0 0x7f604c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6064009b40 tx=0x7f6064009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:20.937 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:20.938+0000 7f605dffb640 1 -- 172.21.15.135:0/3200529790 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f606004d020 con 0x7f606815cc60 2024-04-17T06:10:21.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.128+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6068074940 con 0x7f606815cc60 2024-04-17T06:10:21.743 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.745+0000 7f605dffb640 1 -- 172.21.15.135:0/3200529790 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) ==== 97+0+0 (secure 0 0 0) 0x7f60600ae0c0 con 0x7f606815cc60 2024-04-17T06:10:21.744 INFO:teuthology.orchestra.run.smithi135.stderr: removed all blocklist entries 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f604c06c8f0 msgr2=0x7f604c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f604c06c8f0 0x7f604c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6064009b40 tx=0x7f6064009190 comp rx=0 tx=0).stop 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f606815cc60 msgr2=0x7f606809fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f606815cc60 0x7f606809fd50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6060009f40 tx=0x7f6060004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 shutdown_connections 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f604c06c8f0 0x7f604c06edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f606815cc60 0x7f606809fd50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60680c78d0 0x7f6068153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 --2- 172.21.15.135:0/3200529790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f606809a9f0 0x7f60680b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.747 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.748+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 >> 172.21.15.135:0/3200529790 conn(0x7f60680b41a0 msgr2=0x7f60680c7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:21.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.749+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 shutdown_connections 2024-04-17T06:10:21.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.749+0000 7f606f5db640 1 -- 172.21.15.135:0/3200529790 wait complete. 2024-04-17T06:10:21.760 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-17T06:10:21.867 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.869+0000 7fd247752640 1 Processor -- start 2024-04-17T06:10:21.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.870+0000 7fd247752640 1 -- start start 2024-04-17T06:10:21.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.870+0000 7fd247752640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd24015bb00 0x7fd2400b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.870+0000 7fd247752640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd2400afdd0 0x7fd2400ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.870+0000 7fd247752640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2400b3a50 con 0x7fd2400b3590 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.870+0000 7fd247752640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd2400b4660 con 0x7fd2400afdd0 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.870+0000 7fd247752640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fd2400b3d30 con 0x7fd24015bb00 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd245cef640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd2400afdd0 0x7fd2400ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd245cef640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd2400afdd0 0x7fd2400ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42360/0 (socket says 172.21.15.135:42360) 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd245cef640 1 -- 172.21.15.135:0/3061392819 learned_addr learned my addr 172.21.15.135:0/3061392819 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd245cef640 1 -- 172.21.15.135:0/3061392819 >> v2:172.21.15.190:3300/0 conn(0x7fd24015bb00 msgr2=0x7fd2400b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd2454ee640 1 --2- 172.21.15.135:0/3061392819 >> v2:172.21.15.190:3300/0 conn(0x7fd24015bb00 0x7fd2400b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd245cef640 1 --2- 172.21.15.135:0/3061392819 >> v2:172.21.15.190:3300/0 conn(0x7fd24015bb00 0x7fd2400b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd245cef640 1 -- 172.21.15.135:0/3061392819 >> v1:172.21.15.190:6789/0 conn(0x7fd2400b3590 legacy=0x7fd2400af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd245cef640 1 -- 172.21.15.135:0/3061392819 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2400b4010 con 0x7fd2400afdd0 2024-04-17T06:10:21.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.871+0000 7fd2454ee640 1 --2- 172.21.15.135:0/3061392819 >> v2:172.21.15.190:3300/0 conn(0x7fd24015bb00 0x7fd2400b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd245cef640 1 --2- 172.21.15.135:0/3061392819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400afdd0 0x7fd2400ce720 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd24006dc60 tx=0x7fd23802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec328a6fa376d725 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd237fff640 1 -- 172.21.15.135:0/3061392819 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23802ca80 con 0x7fd2400afdd0 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd237fff640 1 -- 172.21.15.135:0/3061392819 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23802cc20 con 0x7fd2400afdd0 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd237fff640 1 -- 172.21.15.135:0/3061392819 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380343f0 con 0x7fd2400afdd0 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd247752640 1 -- 172.21.15.135:0/3061392819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400afdd0 msgr2=0x7fd2400ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd247752640 1 --2- 172.21.15.135:0/3061392819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400afdd0 0x7fd2400ce720 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd24006dc60 tx=0x7fd23802bee0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd247752640 1 -- 172.21.15.135:0/3061392819 shutdown_connections 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd247752640 1 --2- 172.21.15.135:0/3061392819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400afdd0 0x7fd2400ce720 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd247752640 1 --2- 172.21.15.135:0/3061392819 >> v2:172.21.15.190:3300/0 conn(0x7fd24015bb00 0x7fd2400b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd247752640 1 -- 172.21.15.135:0/3061392819 >> 172.21.15.135:0/3061392819 conn(0x7fd2400d0520 msgr2=0x7fd2400d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd247752640 1 -- 172.21.15.135:0/3061392819 shutdown_connections 2024-04-17T06:10:21.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.872+0000 7fd247752640 1 -- 172.21.15.135:0/3061392819 wait complete. 2024-04-17T06:10:21.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd247752640 1 Processor -- start 2024-04-17T06:10:21.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd247752640 1 -- start start 2024-04-17T06:10:21.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd247752640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2400afdd0 0x7fd240144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:21.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd247752640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400b3590 0x7fd24015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:21.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd247752640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd24015bb00 0x7fd24015bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:21.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd247752640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2400b5810 con 0x7fd2400b3590 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd247752640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd2400b5690 con 0x7fd2400afdd0 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd247752640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd2400cec60 con 0x7fd24015bb00 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd244ced640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400b3590 0x7fd24015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd244ced640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400b3590 0x7fd24015e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42374/0 (socket says 172.21.15.135:42374) 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.874+0000 7fd244ced640 1 -- 172.21.15.135:0/164967093 learned_addr learned my addr 172.21.15.135:0/164967093 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd2454ee640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2400afdd0 0x7fd240144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd245cef640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd24015bb00 0x7fd24015bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd244ced640 1 -- 172.21.15.135:0/164967093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd24015bb00 msgr2=0x7fd24015bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd244ced640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd24015bb00 0x7fd24015bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd244ced640 1 -- 172.21.15.135:0/164967093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2400afdd0 msgr2=0x7fd240144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd244ced640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2400afdd0 0x7fd240144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd244ced640 1 -- 172.21.15.135:0/164967093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd240147780 con 0x7fd2400b3590 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd244ced640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400b3590 0x7fd24015e6f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd23000c3f0 tx=0x7fd23000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:21.876 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd235ffb640 1 -- 172.21.15.135:0/164967093 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230007020 con 0x7fd2400b3590 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd235ffb640 1 -- 172.21.15.135:0/164967093 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2300071c0 con 0x7fd2400b3590 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd240147cd0 con 0x7fd2400b3590 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd235ffb640 1 -- 172.21.15.135:0/164967093 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2300075b0 con 0x7fd2400b3590 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.875+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd240148290 con 0x7fd2400b3590 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.876+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2400b1a10 con 0x7fd2400b3590 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.877+0000 7fd235ffb640 1 -- 172.21.15.135:0/164967093 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd230019020 con 0x7fd2400b3590 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.877+0000 7fd235ffb640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd22406c8f0 0x7fd22406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.877+0000 7fd235ffb640 1 -- 172.21.15.135:0/164967093 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fd23008b040 con 0x7fd2400b3590 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.877+0000 7fd2454ee640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd22406c8f0 0x7fd22406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:21.877 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.878+0000 7fd2454ee640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd22406c8f0 0x7fd22406edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00aac0 tx=0x7fd23c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:21.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:21.881+0000 7fd235ffb640 1 -- 172.21.15.135:0/164967093 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2400b1a10 con 0x7fd2400b3590 2024-04-17T06:10:22.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.125+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd240074940 con 0x7fd2400b3590 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.126+0000 7fd235ffb640 1 -- 172.21.15.135:0/164967093 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7fd230059630 con 0x7fd2400b3590 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout: "auth_dump": [ 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "mds.a", 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBVZx9mwV6rMhAA+B8sT5k/vUBR/B97UZtNYg==", 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow", 2024-04-17T06:10:22.124 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow *", 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow *", 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "mds.b", 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBWZx9m1+31AxAAAF2amMrczR6a53kgr6jvSg==", 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow", 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow *", 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow *", 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.125 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "mds.c", 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBVZx9mdERUOhAASUbdm3i3s/yjZtUy8BnSnQ==", 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow", 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow *", 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow *", 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "mds.d", 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBWZx9mIxuRCxAAu+KJ6Y1OKIdqK2gcpGYPOg==", 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow", 2024-04-17T06:10:22.126 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow *", 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow *", 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "osd.0", 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBZZx9m2rGCKBAAbzZVYc+eHBEsfL6fCbslQg==", 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow profile osd", 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile osd", 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.127 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "osd.1", 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBcZx9mRW5aJxAAzSSCL8AxDpsuTK3DVBC+Cg==", 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow profile osd", 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile osd", 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "osd.2", 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBfZx9m2wwJJhAAtxdZwlcjcDoAloZT4/2FDw==", 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow profile osd", 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile osd", 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.128 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "osd.3", 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBiZx9m5x0oJRAAG93B3QVvH2tREQsjFDdzQA==", 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow profile osd", 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile osd", 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "osd.4", 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBoZx9mThkiMRAAA5EpOIEBihnj6sfsuU1+/Q==", 2024-04-17T06:10:22.129 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow profile osd", 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile osd", 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "osd.5", 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBrZx9m//aSLxAAmQz7rOpw28uMG6aojXJf7g==", 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow profile osd", 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile osd", 2024-04-17T06:10:22.130 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "osd.6", 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBuZx9mLHoFLhAAON/rLUW0eaUbghEPtcoKTw==", 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow profile osd", 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile osd", 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.131 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "osd.7", 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBxZx9mBxBjLBAAkv3QB5M4G6vn7+CXhpfqCQ==", 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow profile osd", 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile osd", 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.0", 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBWZx9mgVoAEBAAlX2QvDOCLGvYpV4AcvC0GQ==", 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow", 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow r", 2024-04-17T06:10:22.132 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow rw", 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow rwx" 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.1", 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBWZx9mBg0fFxAAl/vQU1o5OV8YYnz+IQu+oA==", 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow", 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow r", 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow rw", 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow rwx" 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.133 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.2", 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBWZx9mXZ6IExAA8kTDkMxm2R3pKSsR9sv90g==", 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow", 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow r", 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow rw", 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow rwx" 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.3", 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBWZx9mb8q6GhAAHgO6FH2ZBUnmOYdECsUF2Q==", 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow", 2024-04-17T06:10:22.134 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow r", 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow rw", 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow rwx" 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.admin", 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBVZx9mq4dKFBAAzRywZK7Nb6NkmakqaA8pwA==", 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow *", 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "mgr": "allow *", 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow *", 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.135 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.bootstrap-mds", 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQB3Zx9mZhM+KhAA5DLcAIHmRq50VOo3g+GLwg==", 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile bootstrap-mds" 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.bootstrap-mgr", 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQB3Zx9mjhs+KhAABWqzzxxV8rNeeWYVQETuAg==", 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.136 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.bootstrap-osd", 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQB3Zx9miyI+KhAAiugtjZvpf1Cz/dY5QuI3ug==", 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile bootstrap-osd" 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.bootstrap-rbd", 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQB3Zx9mASk+KhAA8aDKBkruOJYNhfH7inJxdg==", 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.137 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQB3Zx9moi8+KhAA80zTnOk0ijMMq+nyPT07Qw==", 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "client.bootstrap-rgw", 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQB3Zx9mjjY+KhAAKg1FnYzWOROOiia+psQBJg==", 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.138 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "mgr.x", 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBVZx9mKqoXLxAAgdNLvJfyvFLBM4ZtNb44PQ==", 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow *", 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile mgr", 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "entity": "mgr.y", 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "key": "AQBVZx9mag0UKhAAZErsmKPrmQnltFsGEY3G0Q==", 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": { 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": "allow *", 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "mon": "allow profile mgr", 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: "osd": "allow *" 2024-04-17T06:10:22.139 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.140 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:22.140 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-17T06:10:22.140 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-17T06:10:22.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd22406c8f0 msgr2=0x7fd22406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd22406c8f0 0x7fd22406edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00aac0 tx=0x7fd23c008040 comp rx=0 tx=0).stop 2024-04-17T06:10:22.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400b3590 msgr2=0x7fd24015e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.141 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400b3590 0x7fd24015e6f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd23000c3f0 tx=0x7fd23000c8b0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 shutdown_connections 2024-04-17T06:10:22.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd24015bb00 0x7fd24015bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd22406c8f0 0x7fd22406edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2400b3590 0x7fd24015e6f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 --2- 172.21.15.135:0/164967093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd2400afdd0 0x7fd240144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 >> 172.21.15.135:0/164967093 conn(0x7fd2400d0520 msgr2=0x7fd24015abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:22.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 shutdown_connections 2024-04-17T06:10:22.142 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.128+0000 7fd247752640 1 -- 172.21.15.135:0/164967093 wait complete. 2024-04-17T06:10:22.142 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-17T06:10:22.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.249+0000 7fcde8d59640 1 Processor -- start 2024-04-17T06:10:22.247 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.249+0000 7fcde8d59640 1 -- start start 2024-04-17T06:10:22.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.249+0000 7fcde8d59640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde415ec00 0x7fcde40d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.248 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.249+0000 7fcde8d59640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcde40d19e0 0x7fcde40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.249+0000 7fcde8d59640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde40d3330 con 0x7fcde40d43d0 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.249+0000 7fcde8d59640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fcde415efe0 con 0x7fcde40d19e0 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.249+0000 7fcde8d59640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcde40d3f40 con 0x7fcde415ec00 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.250+0000 7fcde2d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcde40d19e0 0x7fcde40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.250+0000 7fcde2d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcde40d19e0 0x7fcde40b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42388/0 (socket says 172.21.15.135:42388) 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.250+0000 7fcde2d9d640 1 -- 172.21.15.135:0/238328834 learned_addr learned my addr 172.21.15.135:0/238328834 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.250+0000 7fcde2d9d640 1 -- 172.21.15.135:0/238328834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde415ec00 msgr2=0x7fcde40d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.250+0000 7fcde2d9d640 1 --2- 172.21.15.135:0/238328834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde415ec00 0x7fcde40d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.250+0000 7fcde2d9d640 1 -- 172.21.15.135:0/238328834 >> v1:172.21.15.190:6789/0 conn(0x7fcde40d43d0 legacy=0x7fcde40d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.250+0000 7fcde2d9d640 1 -- 172.21.15.135:0/238328834 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde40d3610 con 0x7fcde40d19e0 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde2d9d640 1 --2- 172.21.15.135:0/238328834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde40d19e0 0x7fcde40b7a40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009e50 tx=0x7fcddc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542c1aafe66923d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde1d9b640 1 -- 172.21.15.135:0/238328834 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc02cb00 con 0x7fcde40d19e0 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde1d9b640 1 -- 172.21.15.135:0/238328834 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcddc02cca0 con 0x7fcde40d19e0 2024-04-17T06:10:22.249 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde1d9b640 1 -- 172.21.15.135:0/238328834 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc034610 con 0x7fcde40d19e0 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde8d59640 1 -- 172.21.15.135:0/238328834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde40d19e0 msgr2=0x7fcde40b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde8d59640 1 --2- 172.21.15.135:0/238328834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde40d19e0 0x7fcde40b7a40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009e50 tx=0x7fcddc02bf60 comp rx=0 tx=0).stop 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde8d59640 1 -- 172.21.15.135:0/238328834 shutdown_connections 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde8d59640 1 --2- 172.21.15.135:0/238328834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde40d19e0 0x7fcde40b7a40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde8d59640 1 --2- 172.21.15.135:0/238328834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde415ec00 0x7fcde40d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde8d59640 1 -- 172.21.15.135:0/238328834 >> 172.21.15.135:0/238328834 conn(0x7fcde406cc90 msgr2=0x7fcde406d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde8d59640 1 -- 172.21.15.135:0/238328834 shutdown_connections 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.251+0000 7fcde8d59640 1 -- 172.21.15.135:0/238328834 wait complete. 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.253+0000 7fcde8d59640 1 Processor -- start 2024-04-17T06:10:22.251 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.253+0000 7fcde8d59640 1 -- start start 2024-04-17T06:10:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.253+0000 7fcde8d59640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde40d19e0 0x7fcde414bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.253+0000 7fcde8d59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde40d43d0 0x7fcde40cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.253+0000 7fcde8d59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde415ec00 0x7fcde40ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.253+0000 7fcde8d59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcde40d2ed0 con 0x7fcde415ec00 2024-04-17T06:10:22.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.253+0000 7fcde8d59640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcde40d3bd0 con 0x7fcde40d43d0 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.253+0000 7fcde8d59640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcde40d3050 con 0x7fcde40d19e0 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde415ec00 0x7fcde40ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde415ec00 0x7fcde40ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42390/0 (socket says 172.21.15.135:42390) 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 -- 172.21.15.135:0/1390133316 learned_addr learned my addr 172.21.15.135:0/1390133316 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde259c640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde40d19e0 0x7fcde414bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcddbfff640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde40d43d0 0x7fcde40cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 -- 172.21.15.135:0/1390133316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde40d19e0 msgr2=0x7fcde414bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde40d19e0 0x7fcde414bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 -- 172.21.15.135:0/1390133316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde40d43d0 msgr2=0x7fcde40cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde40d43d0 0x7fcde40cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 -- 172.21.15.135:0/1390133316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde40cdad0 con 0x7fcde415ec00 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde2d9d640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde415ec00 0x7fcde40ca3a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009e50 tx=0x7fcddc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:22.254 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcddaffd640 1 -- 172.21.15.135:0/1390133316 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc043020 con 0x7fcde415ec00 2024-04-17T06:10:22.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde40cddb0 con 0x7fcde415ec00 2024-04-17T06:10:22.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcddaffd640 1 -- 172.21.15.135:0/1390133316 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcddc02ce50 con 0x7fcde415ec00 2024-04-17T06:10:22.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcddaffd640 1 -- 172.21.15.135:0/1390133316 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc03e670 con 0x7fcde415ec00 2024-04-17T06:10:22.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.254+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde40ce370 con 0x7fcde415ec00 2024-04-17T06:10:22.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.255+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde4074940 con 0x7fcde415ec00 2024-04-17T06:10:22.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.256+0000 7fcddaffd640 1 -- 172.21.15.135:0/1390133316 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcddc046020 con 0x7fcde415ec00 2024-04-17T06:10:22.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.256+0000 7fcddaffd640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcdc406c8f0 0x7fcdc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.256+0000 7fcde259c640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcdc406c8f0 0x7fcdc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.256+0000 7fcddaffd640 1 -- 172.21.15.135:0/1390133316 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fcddc0ae0c0 con 0x7fcde415ec00 2024-04-17T06:10:22.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.257+0000 7fcde259c640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcdc406c8f0 0x7fcdc406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcde40b84a0 tx=0x7fcdd4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:22.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.261+0000 7fcddaffd640 1 -- 172.21.15.135:0/1390133316 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcddc04d030 con 0x7fcde415ec00 2024-04-17T06:10:22.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.506+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fcde415aeb0 con 0x7fcde415ec00 2024-04-17T06:10:22.506 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.508+0000 7fcddaffd640 1 -- 172.21.15.135:0/1390133316 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fcddc07c740 con 0x7fcde415ec00 2024-04-17T06:10:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcdc406c8f0 msgr2=0x7fcdc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.508 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcdc406c8f0 0x7fcdc406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcde40b84a0 tx=0x7fcdd4008040 comp rx=0 tx=0).stop 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde415ec00 msgr2=0x7fcde40ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde415ec00 0x7fcde40ca3a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009e50 tx=0x7fcddc004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 shutdown_connections 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcdc406c8f0 0x7fcdc406edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde415ec00 0x7fcde40ca3a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcde40d43d0 0x7fcde40cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 --2- 172.21.15.135:0/1390133316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcde40d19e0 0x7fcde414bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 >> 172.21.15.135:0/1390133316 conn(0x7fcde406cc90 msgr2=0x7fcde40b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 shutdown_connections 2024-04-17T06:10:22.509 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.510+0000 7fcde8d59640 1 -- 172.21.15.135:0/1390133316 wait complete. 2024-04-17T06:10:22.520 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-17T06:10:22.624 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.626+0000 7f4787f1c640 1 Processor -- start 2024-04-17T06:10:22.625 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.627+0000 7f4787f1c640 1 -- start start 2024-04-17T06:10:22.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.627+0000 7f4787f1c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f47800d43d0 0x7f47800d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.627+0000 7f4787f1c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47800d1810 0x7f47800b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.627+0000 7f4787f1c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47800d30a0 con 0x7f478015ec00 2024-04-17T06:10:22.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.627+0000 7f4787f1c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f478015efe0 con 0x7f47800d43d0 2024-04-17T06:10:22.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.627+0000 7f4787f1c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f47800d3f40 con 0x7f47800d1810 2024-04-17T06:10:22.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47854b7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f47800d43d0 0x7f47800d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.626 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47854b7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f47800d43d0 0x7f47800d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42402/0 (socket says 172.21.15.135:42402) 2024-04-17T06:10:22.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47854b7640 1 -- 172.21.15.135:0/3300718812 learned_addr learned my addr 172.21.15.135:0/3300718812 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:22.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47864b9640 1 --2- 172.21.15.135:0/3300718812 >> v2:172.21.15.190:3300/0 conn(0x7f47800d1810 0x7f47800b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47854b7640 1 -- 172.21.15.135:0/3300718812 >> v2:172.21.15.190:3300/0 conn(0x7f47800d1810 msgr2=0x7f47800b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:22.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47854b7640 1 --2- 172.21.15.135:0/3300718812 >> v2:172.21.15.190:3300/0 conn(0x7f47800d1810 0x7f47800b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47854b7640 1 -- 172.21.15.135:0/3300718812 >> v1:172.21.15.190:6789/0 conn(0x7f478015ec00 legacy=0x7f47800d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47854b7640 1 -- 172.21.15.135:0/3300718812 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47800d3330 con 0x7f47800d43d0 2024-04-17T06:10:22.627 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.628+0000 7f47864b9640 1 --2- 172.21.15.135:0/3300718812 >> v2:172.21.15.190:3300/0 conn(0x7f47800d1810 0x7f47800b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:22.628 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f47854b7640 1 --2- 172.21.15.135:0/3300718812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f47800d12d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f478006dab0 tx=0x7f477c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bdf49f9117b534 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4784cb6640 1 -- 172.21.15.135:0/3300718812 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c02f850 con 0x7f47800d43d0 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4784cb6640 1 -- 172.21.15.135:0/3300718812 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c02f9f0 con 0x7f47800d43d0 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4784cb6640 1 -- 172.21.15.135:0/3300718812 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c02fdc0 con 0x7f47800d43d0 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4787f1c640 1 -- 172.21.15.135:0/3300718812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 msgr2=0x7f47800d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4787f1c640 1 --2- 172.21.15.135:0/3300718812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f47800d12d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f478006dab0 tx=0x7f477c02ba30 comp rx=0 tx=0).stop 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4787f1c640 1 -- 172.21.15.135:0/3300718812 shutdown_connections 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4787f1c640 1 --2- 172.21.15.135:0/3300718812 >> v2:172.21.15.190:3300/0 conn(0x7f47800d1810 0x7f47800b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4787f1c640 1 --2- 172.21.15.135:0/3300718812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f47800d12d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4787f1c640 1 -- 172.21.15.135:0/3300718812 >> 172.21.15.135:0/3300718812 conn(0x7f478006cc90 msgr2=0x7f478006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.629+0000 7f4787f1c640 1 -- 172.21.15.135:0/3300718812 shutdown_connections 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.630+0000 7f4787f1c640 1 -- 172.21.15.135:0/3300718812 wait complete. 2024-04-17T06:10:22.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f4787f1c640 1 Processor -- start 2024-04-17T06:10:22.630 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f4787f1c640 1 -- start start 2024-04-17T06:10:22.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f4787f1c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800d1810 0x7f478015fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f4787f1c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f4780161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f4787f1c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015ec00 0x7f47800c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.632 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f4787f1c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47800d38f0 con 0x7f47800d43d0 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f4787f1c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f47800d1c60 con 0x7f47800d1810 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f4787f1c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f47800d3bd0 con 0x7f478015ec00 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f47854b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f4780161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f47854b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f4780161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42416/0 (socket says 172.21.15.135:42416) 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.631+0000 7f47854b7640 1 -- 172.21.15.135:0/701759338 learned_addr learned my addr 172.21.15.135:0/701759338 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47864b9640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015ec00 0x7f47800c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f4785cb8640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800d1810 0x7f478015fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47854b7640 1 -- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015ec00 msgr2=0x7f47800c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47854b7640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015ec00 0x7f47800c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47854b7640 1 -- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800d1810 msgr2=0x7f478015fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47854b7640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800d1810 0x7f478015fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47854b7640 1 -- 172.21.15.135:0/701759338 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47800c84e0 con 0x7f47800d43d0 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f4785cb8640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800d1810 0x7f478015fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:22.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47854b7640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f4780161090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f477c000c00 tx=0x7f477c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47727fc640 1 -- 172.21.15.135:0/701759338 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c047070 con 0x7f47800d43d0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47727fc640 1 -- 172.21.15.135:0/701759338 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f477c037cb0 con 0x7f47800d43d0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47800c87c0 con 0x7f47800d43d0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f47727fc640 1 -- 172.21.15.135:0/701759338 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c037e60 con 0x7f47800d43d0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.632+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47800c9060 con 0x7f47800d43d0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.633+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47800c8990 con 0x7f47800d43d0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.633+0000 7f47727fc640 1 -- 172.21.15.135:0/701759338 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f477c04a070 con 0x7f47800d43d0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.634+0000 7f47727fc640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f476406c8f0 0x7f476406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.634+0000 7f47727fc640 1 -- 172.21.15.135:0/701759338 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f477c0adda0 con 0x7f47800d43d0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.634+0000 7f4785cb8640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f476406c8f0 0x7f476406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:22.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.634+0000 7f4785cb8640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f476406c8f0 0x7f476406edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f47740015e0 tx=0x7f4774009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:22.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.638+0000 7f47727fc640 1 -- 172.21.15.135:0/701759338 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f477c033030 con 0x7f47800d43d0 2024-04-17T06:10:22.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.883+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f4780074940 con 0x7f47800d43d0 2024-04-17T06:10:22.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.885+0000 7f47727fc640 1 -- 172.21.15.135:0/701759338 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f477c07c390 con 0x7f47800d43d0 2024-04-17T06:10:22.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f476406c8f0 msgr2=0x7f476406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f476406c8f0 0x7f476406edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f47740015e0 tx=0x7f4774009040 comp rx=0 tx=0).stop 2024-04-17T06:10:22.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 msgr2=0x7f4780161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f4780161090 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f477c000c00 tx=0x7f477c004730 comp rx=0 tx=0).stop 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 shutdown_connections 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f478015ec00 0x7f47800c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f476406c8f0 0x7f476406edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47800d43d0 0x7f4780161090 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 --2- 172.21.15.135:0/701759338 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800d1810 0x7f478015fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 >> 172.21.15.135:0/701759338 conn(0x7f478006cc90 msgr2=0x7f47800b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 shutdown_connections 2024-04-17T06:10:22.886 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:22.887+0000 7f4787f1c640 1 -- 172.21.15.135:0/701759338 wait complete. 2024-04-17T06:10:22.896 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-17T06:10:23.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.003+0000 7f99c6166640 1 Processor -- start 2024-04-17T06:10:23.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.003+0000 7f99c6166640 1 -- start start 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.003+0000 7f99c6166640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015ec00 0x7f99c00d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.003+0000 7f99c6166640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99c00d43d0 0x7f99c00d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.003+0000 7f99c6166640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f99c00d1810 0x7f99c00730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.003+0000 7f99c6166640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f99c015efe0 con 0x7f99c00d43d0 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.003+0000 7f99c6166640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f99c00d3f40 con 0x7f99c00d1810 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.003+0000 7f99c6166640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f99c00d1c70 con 0x7f99c015ec00 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99b6ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99c00d43d0 0x7f99c00d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99b6ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f99c00d43d0 0x7f99c00d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42428/0 (socket says 172.21.15.135:42428) 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99b6ffd640 1 -- 172.21.15.135:0/3257216654 learned_addr learned my addr 172.21.15.135:0/3257216654 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:23.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99bf7fe640 1 --2- 172.21.15.135:0/3257216654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015ec00 0x7f99c00d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99bffff640 1 --2- 172.21.15.135:0/3257216654 >> v2:172.21.15.190:3300/0 conn(0x7f99c00d1810 0x7f99c00730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99b6ffd640 1 -- 172.21.15.135:0/3257216654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015ec00 msgr2=0x7f99c00d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99b6ffd640 1 --2- 172.21.15.135:0/3257216654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015ec00 0x7f99c00d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99b6ffd640 1 -- 172.21.15.135:0/3257216654 >> v2:172.21.15.190:3300/0 conn(0x7f99c00d1810 msgr2=0x7f99c00730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99b6ffd640 1 --2- 172.21.15.135:0/3257216654 >> v2:172.21.15.190:3300/0 conn(0x7f99c00d1810 0x7f99c00730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99b6ffd640 1 -- 172.21.15.135:0/3257216654 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c00d3330 con 0x7f99c00d43d0 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99bf7fe640 1 --2- 172.21.15.135:0/3257216654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015ec00 0x7f99c00d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.004+0000 7f99bffff640 1 --2- 172.21.15.135:0/3257216654 >> v2:172.21.15.190:3300/0 conn(0x7f99c00d1810 0x7f99c00730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99b6ffd640 1 --2- 172.21.15.135:0/3257216654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d43d0 0x7f99c00d12d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f99c006dd80 tx=0x7f99a802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2583bd53c1f71e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99beffd640 1 -- 172.21.15.135:0/3257216654 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8034070 con 0x7f99c00d43d0 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99beffd640 1 -- 172.21.15.135:0/3257216654 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a8030860 con 0x7f99c00d43d0 2024-04-17T06:10:23.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99beffd640 1 -- 172.21.15.135:0/3257216654 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8030c30 con 0x7f99c00d43d0 2024-04-17T06:10:23.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 -- 172.21.15.135:0/3257216654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d43d0 msgr2=0x7f99c00d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 --2- 172.21.15.135:0/3257216654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d43d0 0x7f99c00d12d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f99c006dd80 tx=0x7f99a802b9a0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.007 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 -- 172.21.15.135:0/3257216654 shutdown_connections 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 --2- 172.21.15.135:0/3257216654 >> v2:172.21.15.190:3300/0 conn(0x7f99c00d1810 0x7f99c00730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 --2- 172.21.15.135:0/3257216654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d43d0 0x7f99c00d12d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 --2- 172.21.15.135:0/3257216654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015ec00 0x7f99c00d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 -- 172.21.15.135:0/3257216654 >> 172.21.15.135:0/3257216654 conn(0x7f99c006cc90 msgr2=0x7f99c006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 -- 172.21.15.135:0/3257216654 shutdown_connections 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.005+0000 7f99c6166640 1 -- 172.21.15.135:0/3257216654 wait complete. 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.006+0000 7f99c6166640 1 Processor -- start 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.007+0000 7f99c6166640 1 -- start start 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.007+0000 7f99c6166640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d1810 0x7f99c00c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.007+0000 7f99c6166640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00d43d0 0x7f99c0155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.007+0000 7f99c6166640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015ec00 0x7f99c00ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.007+0000 7f99c6166640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99c00d3bd0 con 0x7f99c00d1810 2024-04-17T06:10:23.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.007+0000 7f99c6166640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f99c00d38f0 con 0x7f99c015ec00 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.007+0000 7f99c6166640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f99c00d2ed0 con 0x7f99c00d43d0 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d1810 0x7f99c00c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d1810 0x7f99c00c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42440/0 (socket says 172.21.15.135:42440) 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 -- 172.21.15.135:0/606184761 learned_addr learned my addr 172.21.15.135:0/606184761 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99b6ffd640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00d43d0 0x7f99c0155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bffff640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015ec00 0x7f99c00ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 -- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00d43d0 msgr2=0x7f99c0155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00d43d0 0x7f99c0155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 -- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015ec00 msgr2=0x7f99c00ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015ec00 0x7f99c00ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 -- 172.21.15.135:0/606184761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c00c9350 con 0x7f99c00d1810 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bffff640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015ec00 0x7f99c00ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bf7fe640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d1810 0x7f99c00c8440 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f99b000d800 tx=0x7f99b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bcff9640 1 -- 172.21.15.135:0/606184761 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0004030 con 0x7f99c00d1810 2024-04-17T06:10:23.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bcff9640 1 -- 172.21.15.135:0/606184761 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b00041d0 con 0x7f99c00d1810 2024-04-17T06:10:23.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c00c8ab0 con 0x7f99c00d1810 2024-04-17T06:10:23.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99bcff9640 1 -- 172.21.15.135:0/606184761 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b00096a0 con 0x7f99c00d1810 2024-04-17T06:10:23.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.008+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c00c9070 con 0x7f99c00d1810 2024-04-17T06:10:23.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.009+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c0074940 con 0x7f99c00d1810 2024-04-17T06:10:23.010 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.010+0000 7f99bcff9640 1 -- 172.21.15.135:0/606184761 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99b0009840 con 0x7f99c00d1810 2024-04-17T06:10:23.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.010+0000 7f99bcff9640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f99a006c920 0x7f99a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.010+0000 7f99bcff9640 1 -- 172.21.15.135:0/606184761 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f99b008bba0 con 0x7f99c00d1810 2024-04-17T06:10:23.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.010+0000 7f99b6ffd640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f99a006c920 0x7f99a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.011 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.011+0000 7f99b6ffd640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f99a006c920 0x7f99a006ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99a8036000 tx=0x7f99a803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.014+0000 7f99bcff9640 1 -- 172.21.15.135:0/606184761 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f99b0011340 con 0x7f99c00d1810 2024-04-17T06:10:23.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.258+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f99c00c8100 con 0x7f99c00d1810 2024-04-17T06:10:23.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.261+0000 7f99bcff9640 1 -- 172.21.15.135:0/606184761 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f99b005a220 con 0x7f99c00d1810 2024-04-17T06:10:23.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f99a006c920 msgr2=0x7f99a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f99a006c920 0x7f99a006ede0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99a8036000 tx=0x7f99a803b040 comp rx=0 tx=0).stop 2024-04-17T06:10:23.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d1810 msgr2=0x7f99c00c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d1810 0x7f99c00c8440 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f99b000d800 tx=0x7f99b000dcc0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 shutdown_connections 2024-04-17T06:10:23.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015ec00 0x7f99c00ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00d43d0 0x7f99c0155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f99a006c920 0x7f99a006ede0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 --2- 172.21.15.135:0/606184761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99c00d1810 0x7f99c00c8440 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 >> 172.21.15.135:0/606184761 conn(0x7f99c006cc90 msgr2=0x7f99c00b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:23.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 shutdown_connections 2024-04-17T06:10:23.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.263+0000 7f99c6166640 1 -- 172.21.15.135:0/606184761 wait complete. 2024-04-17T06:10:23.272 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-17T06:10:23.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.378+0000 7f5c5b8c8640 1 Processor -- start 2024-04-17T06:10:23.376 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.378+0000 7f5c5b8c8640 1 -- start start 2024-04-17T06:10:23.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c5b8c8640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5c540cfe10 0x7f5c54169b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c5b8c8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c54160060 con 0x7f5c5415ec70 2024-04-17T06:10:23.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c5b8c8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c540b5ae0 con 0x7f5c5406b990 2024-04-17T06:10:23.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c5b8c8640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5406bdf0 con 0x7f5c540cfe10 2024-04-17T06:10:23.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c59e65640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f5c540cfe10 0x7f5c54169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.377 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c59664640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f5c5415ec70 0x7f5c540d0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46422/0 (socket says 172.21.15.135:46422) 2024-04-17T06:10:23.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c59664640 1 -- 172.21.15.135:0/3993685841 learned_addr learned my addr 172.21.15.135:0/3993685841 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:23.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c59e65640 1 -- 172.21.15.135:0/3993685841 >> v1:172.21.15.190:6789/0 conn(0x7f5c5406b990 legacy=0x7f5c540cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c59e65640 1 -- 172.21.15.135:0/3993685841 >> v1:172.21.15.135:6789/0 conn(0x7f5c5415ec70 legacy=0x7f5c540d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.378 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.379+0000 7f5c59e65640 1 -- 172.21.15.135:0/3993685841 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c540b5dc0 con 0x7f5c540cfe10 2024-04-17T06:10:23.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c59e65640 1 --2- 172.21.15.135:0/3993685841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c540cfe10 0x7f5c54169b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009d90 tx=0x7f5c4c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aa2c8535fa90a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c47fff640 1 -- 172.21.15.135:0/3993685841 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c039070 con 0x7f5c540cfe10 2024-04-17T06:10:23.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c47fff640 1 -- 172.21.15.135:0/3993685841 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c02cc60 con 0x7f5c540cfe10 2024-04-17T06:10:23.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c47fff640 1 -- 172.21.15.135:0/3993685841 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c034660 con 0x7f5c540cfe10 2024-04-17T06:10:23.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3993685841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c540cfe10 msgr2=0x7f5c54169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c5b8c8640 1 --2- 172.21.15.135:0/3993685841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c540cfe10 0x7f5c54169b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009d90 tx=0x7f5c4c02beb0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3993685841 shutdown_connections 2024-04-17T06:10:23.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c5b8c8640 1 --2- 172.21.15.135:0/3993685841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c540cfe10 0x7f5c54169b20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.380+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3993685841 >> 172.21.15.135:0/3993685841 conn(0x7f5c540cd7e0 msgr2=0x7f5c540cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:23.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.381+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3993685841 shutdown_connections 2024-04-17T06:10:23.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.381+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3993685841 wait complete. 2024-04-17T06:10:23.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c5b8c8640 1 Processor -- start 2024-04-17T06:10:23.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c5b8c8640 1 -- start start 2024-04-17T06:10:23.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c5b8c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c5406b990 0x7f5c5406f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c5b8c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c540cfe10 0x7f5c541ea790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c5b8c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c5415ec70 0x7f5c541f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c5b8c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5c540b4160 con 0x7f5c5415ec70 2024-04-17T06:10:23.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c5b8c8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f5c540b60a0 con 0x7f5c540cfe10 2024-04-17T06:10:23.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c5b8c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f5c540d0240 con 0x7f5c5406b990 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.382+0000 7f5c59e65640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c5415ec70 0x7f5c541f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59e65640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c5415ec70 0x7f5c541f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42464/0 (socket says 172.21.15.135:42464) 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59e65640 1 -- 172.21.15.135:0/3365715421 learned_addr learned my addr 172.21.15.135:0/3365715421 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c58e63640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c540cfe10 0x7f5c541ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59664640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c5406b990 0x7f5c5406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59e65640 1 -- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c5406b990 msgr2=0x7f5c5406f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59e65640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c5406b990 0x7f5c5406f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59e65640 1 -- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c540cfe10 msgr2=0x7f5c541ea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59e65640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c540cfe10 0x7f5c541ea790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59e65640 1 -- 172.21.15.135:0/3365715421 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c541eb330 con 0x7f5c5415ec70 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c58e63640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c540cfe10 0x7f5c541ea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c59e65640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c5415ec70 0x7f5c541f8490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009d90 tx=0x7f5c4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c45ffb640 1 -- 172.21.15.135:0/3365715421 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c039050 con 0x7f5c5415ec70 2024-04-17T06:10:23.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c45ffb640 1 -- 172.21.15.135:0/3365715421 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c045070 con 0x7f5c5415ec70 2024-04-17T06:10:23.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5415d5c0 con 0x7f5c5415ec70 2024-04-17T06:10:23.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c45ffb640 1 -- 172.21.15.135:0/3365715421 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c03e830 con 0x7f5c5415ec70 2024-04-17T06:10:23.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.383+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c541e7340 con 0x7f5c5415ec70 2024-04-17T06:10:23.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.384+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c54074940 con 0x7f5c5415ec70 2024-04-17T06:10:23.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.385+0000 7f5c45ffb640 1 -- 172.21.15.135:0/3365715421 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c4c03e9d0 con 0x7f5c5415ec70 2024-04-17T06:10:23.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.385+0000 7f5c45ffb640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f5c3806c8f0 0x7f5c3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.385+0000 7f5c45ffb640 1 -- 172.21.15.135:0/3365715421 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f5c4c0ae790 con 0x7f5c5415ec70 2024-04-17T06:10:23.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.385+0000 7f5c59664640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f5c3806c8f0 0x7f5c3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.386+0000 7f5c59664640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f5c3806c8f0 0x7f5c3806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005880 tx=0x7f5c48009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.390+0000 7f5c45ffb640 1 -- 172.21.15.135:0/3365715421 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c4c07cd80 con 0x7f5c5415ec70 2024-04-17T06:10:23.629 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.631+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f5c5415dae0 con 0x7f5c5415ec70 2024-04-17T06:10:23.631 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.633+0000 7f5c45ffb640 1 -- 172.21.15.135:0/3365715421 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f5c4c07c680 con 0x7f5c5415ec70 2024-04-17T06:10:23.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f5c3806c8f0 msgr2=0x7f5c3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f5c3806c8f0 0x7f5c3806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005880 tx=0x7f5c48009040 comp rx=0 tx=0).stop 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c5415ec70 msgr2=0x7f5c541f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c5415ec70 0x7f5c541f8490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009d90 tx=0x7f5c4c004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 shutdown_connections 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f5c3806c8f0 0x7f5c3806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5c5415ec70 0x7f5c541f8490 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c540cfe10 0x7f5c541ea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 --2- 172.21.15.135:0/3365715421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c5406b990 0x7f5c5406f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 >> 172.21.15.135:0/3365715421 conn(0x7f5c540cd7e0 msgr2=0x7f5c540cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 shutdown_connections 2024-04-17T06:10:23.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.635+0000 7f5c5b8c8640 1 -- 172.21.15.135:0/3365715421 wait complete. 2024-04-17T06:10:23.644 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-17T06:10:23.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.750+0000 7f80e5e8f640 1 Processor -- start 2024-04-17T06:10:23.749 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.750+0000 7f80e5e8f640 1 -- start start 2024-04-17T06:10:23.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80e5e8f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0144f70 0x7f80e00cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80e5e8f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80e015f500 0x7f80e015f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.751 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80e5e8f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f80e0073040 0x7f80e0073490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80e5e8f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f80e00cf130 con 0x7f80e0073040 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80e5e8f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f80e00739d0 con 0x7f80e015f500 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80e5e8f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80e00b75c0 con 0x7f80e0144f70 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80dffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f80e0073040 0x7f80e0073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80dffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f80e0073040 0x7f80e0073490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42480/0 (socket says 172.21.15.135:42480) 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80dffff640 1 -- 172.21.15.135:0/3781005654 learned_addr learned my addr 172.21.15.135:0/3781005654 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80deffd640 1 --2- 172.21.15.135:0/3781005654 >> v2:172.21.15.190:3300/0 conn(0x7f80e015f500 0x7f80e015f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.751+0000 7f80df7fe640 1 --2- 172.21.15.135:0/3781005654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0144f70 0x7f80e00cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.752+0000 7f80dffff640 1 -- 172.21.15.135:0/3781005654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0144f70 msgr2=0x7f80e00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.752+0000 7f80dffff640 1 --2- 172.21.15.135:0/3781005654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0144f70 0x7f80e00cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.752 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.752+0000 7f80dffff640 1 -- 172.21.15.135:0/3781005654 >> v2:172.21.15.190:3300/0 conn(0x7f80e015f500 msgr2=0x7f80e015f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.752+0000 7f80dffff640 1 --2- 172.21.15.135:0/3781005654 >> v2:172.21.15.190:3300/0 conn(0x7f80e015f500 0x7f80e015f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.752+0000 7f80dffff640 1 -- 172.21.15.135:0/3781005654 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e0160070 con 0x7f80e0073040 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.752+0000 7f80dffff640 1 --2- 172.21.15.135:0/3781005654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e0073040 0x7f80e0073490 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f80e006e7f0 tx=0x7f80d802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b420f86170a476a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80de7fc640 1 -- 172.21.15.135:0/3781005654 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d802c9e0 con 0x7f80e0073040 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80de7fc640 1 -- 172.21.15.135:0/3781005654 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d802cb80 con 0x7f80e0073040 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80de7fc640 1 -- 172.21.15.135:0/3781005654 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d80343f0 con 0x7f80e0073040 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 -- 172.21.15.135:0/3781005654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e0073040 msgr2=0x7f80e0073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/3781005654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e0073040 0x7f80e0073490 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f80e006e7f0 tx=0x7f80d802be40 comp rx=0 tx=0).stop 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 -- 172.21.15.135:0/3781005654 shutdown_connections 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/3781005654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e0073040 0x7f80e0073490 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.753 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/3781005654 >> v2:172.21.15.190:3300/0 conn(0x7f80e015f500 0x7f80e015f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/3781005654 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0144f70 0x7f80e00cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 -- 172.21.15.135:0/3781005654 >> 172.21.15.135:0/3781005654 conn(0x7f80e00d15e0 msgr2=0x7f80e00d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:23.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 -- 172.21.15.135:0/3781005654 shutdown_connections 2024-04-17T06:10:23.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.753+0000 7f80e5e8f640 1 -- 172.21.15.135:0/3781005654 wait complete. 2024-04-17T06:10:23.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.754+0000 7f80e5e8f640 1 Processor -- start 2024-04-17T06:10:23.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80e5e8f640 1 -- start start 2024-04-17T06:10:23.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80e5e8f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0073040 0x7f80e013b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80e5e8f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e0144f70 0x7f80e01407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80e5e8f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e015f500 0x7f80e015dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80e5e8f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80e00ce3f0 con 0x7f80e015f500 2024-04-17T06:10:23.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80e5e8f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f80e00ce110 con 0x7f80e0144f70 2024-04-17T06:10:23.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80e5e8f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f80e0145350 con 0x7f80e0073040 2024-04-17T06:10:23.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80dffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e015f500 0x7f80e015dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80dffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e015f500 0x7f80e015dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42492/0 (socket says 172.21.15.135:42492) 2024-04-17T06:10:23.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.755+0000 7f80dffff640 1 -- 172.21.15.135:0/263442702 learned_addr learned my addr 172.21.15.135:0/263442702 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:23.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80df7fe640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0073040 0x7f80e013b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80deffd640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e0144f70 0x7f80e01407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80dffff640 1 -- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0073040 msgr2=0x7f80e013b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80dffff640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0073040 0x7f80e013b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80dffff640 1 -- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e0144f70 msgr2=0x7f80e01407d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80dffff640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e0144f70 0x7f80e01407d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80dffff640 1 -- 172.21.15.135:0/263442702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e013b850 con 0x7f80e015f500 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80deffd640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e0144f70 0x7f80e01407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80dffff640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e015f500 0x7f80e015dd00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f80d8036040 tx=0x7f80d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80c7fff640 1 -- 172.21.15.135:0/263442702 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8043020 con 0x7f80e015f500 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80c7fff640 1 -- 172.21.15.135:0/263442702 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d803e450 con 0x7f80e015f500 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e0141000 con 0x7f80e015f500 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80c7fff640 1 -- 172.21.15.135:0/263442702 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8009510 con 0x7f80e015f500 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.756+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e01415c0 con 0x7f80e015f500 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.757+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e0068a20 con 0x7f80e015f500 2024-04-17T06:10:23.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.758+0000 7f80c7fff640 1 -- 172.21.15.135:0/263442702 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80d8046020 con 0x7f80e015f500 2024-04-17T06:10:23.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.758+0000 7f80c7fff640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80cc06c920 0x7f80cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:23.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.758+0000 7f80c7fff640 1 -- 172.21.15.135:0/263442702 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f80d80adf00 con 0x7f80e015f500 2024-04-17T06:10:23.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.758+0000 7f80df7fe640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80cc06c920 0x7f80cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:23.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.759+0000 7f80df7fe640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80cc06c920 0x7f80cc06ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f80e0155d00 tx=0x7f80d0009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:23.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:23.763+0000 7f80c7fff640 1 -- 172.21.15.135:0/263442702 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80d804d030 con 0x7f80e015f500 2024-04-17T06:10:24.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.006+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f80e015b950 con 0x7f80e015f500 2024-04-17T06:10:24.006 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.008+0000 7f80c7fff640 1 -- 172.21.15.135:0/263442702 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f80d807c580 con 0x7f80e015f500 2024-04-17T06:10:24.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80cc06c920 msgr2=0x7f80cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80cc06c920 0x7f80cc06ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f80e0155d00 tx=0x7f80d0009190 comp rx=0 tx=0).stop 2024-04-17T06:10:24.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e015f500 msgr2=0x7f80e015dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.008 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e015f500 0x7f80e015dd00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f80d8036040 tx=0x7f80d8004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:24.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 shutdown_connections 2024-04-17T06:10:24.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f80cc06c920 0x7f80cc06ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80e015f500 0x7f80e015dd00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80e0144f70 0x7f80e01407d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 --2- 172.21.15.135:0/263442702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80e0073040 0x7f80e013b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 >> 172.21.15.135:0/263442702 conn(0x7f80e00d15e0 msgr2=0x7f80e00711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:24.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 shutdown_connections 2024-04-17T06:10:24.009 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.010+0000 7f80e5e8f640 1 -- 172.21.15.135:0/263442702 wait complete. 2024-04-17T06:10:24.019 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-17T06:10:24.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.125+0000 7f392b60c640 1 Processor -- start 2024-04-17T06:10:24.123 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.125+0000 7f392b60c640 1 -- start start 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.126+0000 7f392b60c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f392415ee30 0x7f392409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.126+0000 7f392b60c640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f392409bbe0 0x7f392409af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.126+0000 7f392b60c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f392406db80 con 0x7f392409b470 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.126+0000 7f392b60c640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f392406de60 con 0x7f392409bbe0 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.126+0000 7f392b60c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f39240cce30 con 0x7f392415ee30 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.126+0000 7f3928ba7640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f392409bbe0 0x7f392409af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.126+0000 7f3929ba9640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f392409b470 0x7f39240d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:46438/0 (socket says 172.21.15.135:46438) 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.126+0000 7f3929ba9640 1 -- 172.21.15.135:0/3878795078 learned_addr learned my addr 172.21.15.135:0/3878795078 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.127+0000 7f39293a8640 1 --2- 172.21.15.135:0/3878795078 >> v2:172.21.15.190:3300/0 conn(0x7f392415ee30 0x7f392409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.125 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.127+0000 7f3928ba7640 1 -- 172.21.15.135:0/3878795078 >> v2:172.21.15.190:3300/0 conn(0x7f392415ee30 msgr2=0x7f392409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:24.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.127+0000 7f3928ba7640 1 --2- 172.21.15.135:0/3878795078 >> v2:172.21.15.190:3300/0 conn(0x7f392415ee30 0x7f392409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.127+0000 7f3928ba7640 1 -- 172.21.15.135:0/3878795078 >> v1:172.21.15.135:6789/0 conn(0x7f392409b470 legacy=0x7f39240d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.127+0000 7f391bfff640 1 -- 172.21.15.135:0/3878795078 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2301850073 0 0) 0x7f392406db80 con 0x7f392409b470 2024-04-17T06:10:24.126 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.127+0000 7f3928ba7640 1 -- 172.21.15.135:0/3878795078 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240cd110 con 0x7f392409bbe0 2024-04-17T06:10:24.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.127+0000 7f3928ba7640 1 --2- 172.21.15.135:0/3878795078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392409bbe0 0x7f392409af30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3924069720 tx=0x7f391402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359bb19caae568ac server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f391bfff640 1 -- 172.21.15.135:0/3878795078 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914035020 con 0x7f392409bbe0 2024-04-17T06:10:24.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f391bfff640 1 -- 172.21.15.135:0/3878795078 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3914031830 con 0x7f392409bbe0 2024-04-17T06:10:24.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f391bfff640 1 -- 172.21.15.135:0/3878795078 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914031c00 con 0x7f392409bbe0 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f392b60c640 1 -- 172.21.15.135:0/3878795078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392409bbe0 msgr2=0x7f392409af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f392b60c640 1 --2- 172.21.15.135:0/3878795078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392409bbe0 0x7f392409af30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3924069720 tx=0x7f391402b8d0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f392b60c640 1 -- 172.21.15.135:0/3878795078 shutdown_connections 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f392b60c640 1 --2- 172.21.15.135:0/3878795078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392409bbe0 0x7f392409af30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f392b60c640 1 --2- 172.21.15.135:0/3878795078 >> v2:172.21.15.190:3300/0 conn(0x7f392415ee30 0x7f392409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f392b60c640 1 -- 172.21.15.135:0/3878795078 >> 172.21.15.135:0/3878795078 conn(0x7f39240b41a0 msgr2=0x7f39240b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f392b60c640 1 -- 172.21.15.135:0/3878795078 shutdown_connections 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.128+0000 7f392b60c640 1 -- 172.21.15.135:0/3878795078 wait complete. 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.129+0000 7f392b60c640 1 Processor -- start 2024-04-17T06:10:24.128 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f392b60c640 1 -- start start 2024-04-17T06:10:24.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f392b60c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f392409b470 0x7f39240d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f392b60c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f392409bbe0 0x7f39240b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f392b60c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392415ee30 0x7f3924150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f392b60c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f392406d910 con 0x7f392415ee30 2024-04-17T06:10:24.131 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f392b60c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f392406d790 con 0x7f392409bbe0 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f392b60c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f39240ce640 con 0x7f392409b470 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f3929ba9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392415ee30 0x7f3924150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f3929ba9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392415ee30 0x7f3924150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:42520/0 (socket says 172.21.15.135:42520) 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.130+0000 7f3929ba9640 1 -- 172.21.15.135:0/2863754963 learned_addr learned my addr 172.21.15.135:0/2863754963 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3928ba7640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f392409bbe0 0x7f39240b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f39293a8640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f392409b470 0x7f39240d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3929ba9640 1 -- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f392409b470 msgr2=0x7f39240d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3929ba9640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f392409b470 0x7f39240d0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3929ba9640 1 -- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f392409bbe0 msgr2=0x7f39240b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3929ba9640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f392409bbe0 0x7f39240b7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3929ba9640 1 -- 172.21.15.135:0/2863754963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240d14e0 con 0x7f392415ee30 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3928ba7640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f392409bbe0 0x7f39240b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:24.132 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f39293a8640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f392409b470 0x7f39240d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3929ba9640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392415ee30 0x7f3924150720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c7e0 tx=0x7f391c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3919ffb640 1 -- 172.21.15.135:0/2863754963 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c018070 con 0x7f392415ee30 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3919ffb640 1 -- 172.21.15.135:0/2863754963 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391c00a640 con 0x7f392415ee30 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392415f210 con 0x7f392415ee30 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f3919ffb640 1 -- 172.21.15.135:0/2863754963 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c00aa30 con 0x7f392415ee30 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.131+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39241f4d90 con 0x7f392415ee30 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.132+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3924074940 con 0x7f392415ee30 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.133+0000 7f3919ffb640 1 -- 172.21.15.135:0/2863754963 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f391c01b070 con 0x7f392415ee30 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.133+0000 7f3919ffb640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f390806c8f0 0x7f390806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.133+0000 7f3919ffb640 1 -- 172.21.15.135:0/2863754963 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f391c08afa0 con 0x7f392415ee30 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.133+0000 7f39293a8640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f390806c8f0 0x7f390806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.133+0000 7f39293a8640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f390806c8f0 0x7f390806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3920006fd0 tx=0x7f3920008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.137+0000 7f3919ffb640 1 -- 172.21.15.135:0/2863754963 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f391c059590 con 0x7f392415ee30 2024-04-17T06:10:24.379 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.381+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f392415e100 con 0x7f392415ee30 2024-04-17T06:10:24.380 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.382+0000 7f3919ffb640 1 -- 172.21.15.135:0/2863754963 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f391c058e90 con 0x7f392415ee30 2024-04-17T06:10:24.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f390806c8f0 msgr2=0x7f390806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f390806c8f0 0x7f390806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3920006fd0 tx=0x7f3920008040 comp rx=0 tx=0).stop 2024-04-17T06:10:24.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392415ee30 msgr2=0x7f3924150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392415ee30 0x7f3924150720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f391c00c7e0 tx=0x7f391c00cca0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 shutdown_connections 2024-04-17T06:10:24.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f390806c8f0 0x7f390806edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f392415ee30 0x7f3924150720 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f392409bbe0 0x7f39240b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 --2- 172.21.15.135:0/2863754963 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f392409b470 0x7f39240d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 >> 172.21.15.135:0/2863754963 conn(0x7f39240b41a0 msgr2=0x7f392409a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:24.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 shutdown_connections 2024-04-17T06:10:24.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.385+0000 7f392b60c640 1 -- 172.21.15.135:0/2863754963 wait complete. 2024-04-17T06:10:24.394 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:24.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.501+0000 7f6b1f1e2640 1 Processor -- start 2024-04-17T06:10:24.499 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.501+0000 7f6b1f1e2640 1 -- start start 2024-04-17T06:10:24.500 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b1f1e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18162190 0x7f6b1806d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b1f1e2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6b180d1cf0 0x7f6b1809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b1f1e2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b180d2150 con 0x7f6b1809af30 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b1f1e2640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1809b3a0 con 0x7f6b180d1cf0 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b1f1e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6b180ccd00 con 0x7f6b18162190 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b0ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6b180d1cf0 0x7f6b1809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b0ffff640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f6b180d1cf0 0x7f6b1809a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54562/0 (socket says 172.21.15.135:54562) 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b0ffff640 1 -- 172.21.15.135:0/2044558884 learned_addr learned my addr 172.21.15.135:0/2044558884 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b1cf7e640 1 --2- 172.21.15.135:0/2044558884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18162190 0x7f6b1806d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b0ffff640 1 -- 172.21.15.135:0/2044558884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18162190 msgr2=0x7f6b1806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b0ffff640 1 --2- 172.21.15.135:0/2044558884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18162190 0x7f6b1806d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b0ffff640 1 -- 172.21.15.135:0/2044558884 >> v1:172.21.15.190:6789/0 conn(0x7f6b1809af30 legacy=0x7f6b180d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.501 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.502+0000 7f6b0ffff640 1 -- 172.21.15.135:0/2044558884 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1806ddc0 con 0x7f6b180d1cf0 2024-04-17T06:10:24.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.503+0000 7f6b0ffff640 1 --2- 172.21.15.135:0/2044558884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b180d1cf0 0x7f6b1809a9f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009870 tx=0x7f6b0802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b5214faa6c15f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.503+0000 7f6b0f7fe640 1 -- 172.21.15.135:0/2044558884 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08034070 con 0x7f6b180d1cf0 2024-04-17T06:10:24.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.503+0000 7f6b0f7fe640 1 -- 172.21.15.135:0/2044558884 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b08030840 con 0x7f6b180d1cf0 2024-04-17T06:10:24.503 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.503+0000 7f6b0f7fe640 1 -- 172.21.15.135:0/2044558884 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08030c10 con 0x7f6b180d1cf0 2024-04-17T06:10:24.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.504+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/2044558884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b180d1cf0 msgr2=0x7f6b1809a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.504+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/2044558884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b180d1cf0 0x7f6b1809a9f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009870 tx=0x7f6b0802b980 comp rx=0 tx=0).stop 2024-04-17T06:10:24.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.504+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/2044558884 shutdown_connections 2024-04-17T06:10:24.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.504+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/2044558884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b180d1cf0 0x7f6b1809a9f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.504+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/2044558884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b18162190 0x7f6b1806d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.504 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.504+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/2044558884 >> 172.21.15.135:0/2044558884 conn(0x7f6b180b41a0 msgr2=0x7f6b180b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:24.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.504+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/2044558884 shutdown_connections 2024-04-17T06:10:24.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.504+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/2044558884 wait complete. 2024-04-17T06:10:24.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.505+0000 7f6b1f1e2640 1 Processor -- start 2024-04-17T06:10:24.505 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.505+0000 7f6b1f1e2640 1 -- start start 2024-04-17T06:10:24.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1f1e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1809af30 0x7f6b180b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1f1e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180d1cf0 0x7f6b181f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1f1e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b18162190 0x7f6b180b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1f1e2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180cd2a0 con 0x7f6b1809af30 2024-04-17T06:10:24.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1f1e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f6b180ccfc0 con 0x7f6b18162190 2024-04-17T06:10:24.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1f1e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6b18162570 con 0x7f6b180d1cf0 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1809af30 0x7f6b180b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1809af30 0x7f6b180b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54572/0 (socket says 172.21.15.135:54572) 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 -- 172.21.15.135:0/1675739309 learned_addr learned my addr 172.21.15.135:0/1675739309 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 -- 172.21.15.135:0/1675739309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180d1cf0 msgr2=0x7f6b181f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180d1cf0 0x7f6b181f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 -- 172.21.15.135:0/1675739309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b18162190 msgr2=0x7f6b180b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b18162190 0x7f6b180b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 -- 172.21.15.135:0/1675739309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1814d540 con 0x7f6b1809af30 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.506+0000 7f6b1cf7e640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1809af30 0x7f6b180b4580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e8a0 tx=0x7f6b1400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.507+0000 7f6b0d7fa640 1 -- 172.21.15.135:0/1675739309 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14017070 con 0x7f6b1809af30 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.507+0000 7f6b0d7fa640 1 -- 172.21.15.135:0/1675739309 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1400c940 con 0x7f6b1809af30 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.507+0000 7f6b0d7fa640 1 -- 172.21.15.135:0/1675739309 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400cd30 con 0x7f6b1809af30 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.507+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1814d820 con 0x7f6b1809af30 2024-04-17T06:10:24.513 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.507+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1814dde0 con 0x7f6b1809af30 2024-04-17T06:10:24.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.508+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b18074940 con 0x7f6b1809af30 2024-04-17T06:10:24.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.508+0000 7f6b0d7fa640 1 -- 172.21.15.135:0/1675739309 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b1401a070 con 0x7f6b1809af30 2024-04-17T06:10:24.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.509+0000 7f6b0d7fa640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6b0406c920 0x7f6b0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.509+0000 7f6b0d7fa640 1 -- 172.21.15.135:0/1675739309 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f6b1408bbb0 con 0x7f6b1809af30 2024-04-17T06:10:24.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.509+0000 7f6b0ffff640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6b0406c920 0x7f6b0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.509+0000 7f6b0ffff640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6b0406c920 0x7f6b0406ede0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b08036000 tx=0x7f6b0803b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.513+0000 7f6b0d7fa640 1 -- 172.21.15.135:0/1675739309 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b1405a1a0 con 0x7f6b1809af30 2024-04-17T06:10:24.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.688+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b180d2490 con 0x7f6b1809af30 2024-04-17T06:10:24.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.689+0000 7f6b0d7fa640 1 -- 172.21.15.135:0/1675739309 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13212 (secure 0 0 0) 0x7f6b14059aa0 con 0x7f6b1809af30 2024-04-17T06:10:24.687 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:24.687 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":34,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:21.752888+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:24.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6b0406c920 msgr2=0x7f6b0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6b0406c920 0x7f6b0406ede0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b08036000 tx=0x7f6b0803b040 comp rx=0 tx=0).stop 2024-04-17T06:10:24.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1809af30 msgr2=0x7f6b180b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1809af30 0x7f6b180b4580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e8a0 tx=0x7f6b1400ed60 comp rx=0 tx=0).stop 2024-04-17T06:10:24.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 shutdown_connections 2024-04-17T06:10:24.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b18162190 0x7f6b180b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b180d1cf0 0x7f6b181f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f6b0406c920 0x7f6b0406ede0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 --2- 172.21.15.135:0/1675739309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1809af30 0x7f6b180b4580 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 >> 172.21.15.135:0/1675739309 conn(0x7f6b180b41a0 msgr2=0x7f6b18070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:24.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 shutdown_connections 2024-04-17T06:10:24.690 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.691+0000 7f6b1f1e2640 1 -- 172.21.15.135:0/1675739309 wait complete. 2024-04-17T06:10:24.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:10:24.701 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:10:24.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.809+0000 7f9be7235640 1 Processor -- start 2024-04-17T06:10:24.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.809+0000 7f9be7235640 1 -- start start 2024-04-17T06:10:24.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be7235640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9be0158a50 0x7f9be015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.809 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be7235640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9be006f610 0x7f9be015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be7235640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9be015fb80 con 0x7f9be0158e30 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be7235640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f9be015b840 con 0x7f9be006f610 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be7235640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9be015f2e0 con 0x7f9be0158a50 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be57d2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9be006f610 0x7f9be015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be57d2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f9be006f610 0x7f9be015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54580/0 (socket says 172.21.15.135:54580) 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be57d2640 1 -- 172.21.15.135:0/2436672632 learned_addr learned my addr 172.21.15.135:0/2436672632 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be57d2640 1 -- 172.21.15.135:0/2436672632 >> v2:172.21.15.190:3300/0 conn(0x7f9be0158a50 msgr2=0x7f9be015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be4fd1640 1 --2- 172.21.15.135:0/2436672632 >> v2:172.21.15.190:3300/0 conn(0x7f9be0158a50 0x7f9be015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be57d2640 1 --2- 172.21.15.135:0/2436672632 >> v2:172.21.15.190:3300/0 conn(0x7f9be0158a50 0x7f9be015ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be57d2640 1 -- 172.21.15.135:0/2436672632 >> v1:172.21.15.190:6789/0 conn(0x7f9be0158e30 legacy=0x7f9be00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.810+0000 7f9be57d2640 1 -- 172.21.15.135:0/2436672632 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be015f5c0 con 0x7f9be006f610 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.811+0000 7f9be57d2640 1 --2- 172.21.15.135:0/2436672632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be015bcd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9be006ce10 tx=0x7f9bd802e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad7f197e30e53c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.811+0000 7f9bdf7fe640 1 -- 172.21.15.135:0/2436672632 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd802ea90 con 0x7f9be006f610 2024-04-17T06:10:24.810 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.811+0000 7f9bdf7fe640 1 -- 172.21.15.135:0/2436672632 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd802ec30 con 0x7f9be006f610 2024-04-17T06:10:24.811 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.811+0000 7f9bdf7fe640 1 -- 172.21.15.135:0/2436672632 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8009600 con 0x7f9be006f610 2024-04-17T06:10:24.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.812+0000 7f9be7235640 1 -- 172.21.15.135:0/2436672632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 msgr2=0x7f9be015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.812+0000 7f9be7235640 1 --2- 172.21.15.135:0/2436672632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be015bcd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9be006ce10 tx=0x7f9bd802e0f0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.813 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.812+0000 7f9be7235640 1 -- 172.21.15.135:0/2436672632 shutdown_connections 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.812+0000 7f9be7235640 1 --2- 172.21.15.135:0/2436672632 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be015bcd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.812+0000 7f9be7235640 1 --2- 172.21.15.135:0/2436672632 >> v2:172.21.15.190:3300/0 conn(0x7f9be0158a50 0x7f9be015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.812+0000 7f9be7235640 1 -- 172.21.15.135:0/2436672632 >> 172.21.15.135:0/2436672632 conn(0x7f9be00d0520 msgr2=0x7f9be00d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.812+0000 7f9be7235640 1 -- 172.21.15.135:0/2436672632 shutdown_connections 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.812+0000 7f9be7235640 1 -- 172.21.15.135:0/2436672632 wait complete. 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.813+0000 7f9be7235640 1 Processor -- start 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.813+0000 7f9be7235640 1 -- start start 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be7235640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be01507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be7235640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be0158a50 0x7f9be014f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be7235640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9be0158e30 0x7f9be00ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be7235640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9be00b3ca0 con 0x7f9be006f610 2024-04-17T06:10:24.814 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be7235640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9be006faa0 con 0x7f9be0158a50 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be7235640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9be00cfc10 con 0x7f9be0158e30 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be01507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be01507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54590/0 (socket says 172.21.15.135:54590) 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 -- 172.21.15.135:0/1208914692 learned_addr learned my addr 172.21.15.135:0/1208914692 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 -- 172.21.15.135:0/1208914692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9be0158e30 msgr2=0x7f9be00ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9be0158e30 0x7f9be00ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 -- 172.21.15.135:0/1208914692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be0158a50 msgr2=0x7f9be014f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9bdffff640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be0158a50 0x7f9be014f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be0158a50 0x7f9be014f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 -- 172.21.15.135:0/1208914692 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be0150260 con 0x7f9be006f610 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be4fd1640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be01507f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400c820 tx=0x7f9bd400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9bdd7fa640 1 -- 172.21.15.135:0/1208914692 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4015070 con 0x7f9be006f610 2024-04-17T06:10:24.815 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9bdd7fa640 1 -- 172.21.15.135:0/1208914692 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd4004240 con 0x7f9be006f610 2024-04-17T06:10:24.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be0150d30 con 0x7f9be006f610 2024-04-17T06:10:24.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.814+0000 7f9bdd7fa640 1 -- 172.21.15.135:0/1208914692 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400ce10 con 0x7f9be006f610 2024-04-17T06:10:24.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.815+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be00cf650 con 0x7f9be006f610 2024-04-17T06:10:24.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.816+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be00cdf50 con 0x7f9be006f610 2024-04-17T06:10:24.816 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.816+0000 7f9bdd7fa640 1 -- 172.21.15.135:0/1208914692 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9bd4007830 con 0x7f9be006f610 2024-04-17T06:10:24.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.816+0000 7f9bdd7fa640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9bc406c8f0 0x7f9bc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:24.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.817+0000 7f9bdd7fa640 1 -- 172.21.15.135:0/1208914692 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f9bd408b870 con 0x7f9be006f610 2024-04-17T06:10:24.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.817+0000 7f9bdffff640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9bc406c8f0 0x7f9bc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:24.817 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.817+0000 7f9bdffff640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9bc406c8f0 0x7f9bc406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9be01f3a80 tx=0x7f9bcc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:24.818 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.821+0000 7f9bdd7fa640 1 -- 172.21.15.135:0/1208914692 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9be0150260 con 0x7f9be006f610 2024-04-17T06:10:24.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.995+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9be0074940 con 0x7f9be006f610 2024-04-17T06:10:24.993 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.995+0000 7f9bdd7fa640 1 -- 172.21.15.135:0/1208914692 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f9bd4059e60 con 0x7f9be006f610 2024-04-17T06:10:24.993 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:10:24.995 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9bc406c8f0 msgr2=0x7f9bc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9bc406c8f0 0x7f9bc406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9be01f3a80 tx=0x7f9bcc008040 comp rx=0 tx=0).stop 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 msgr2=0x7f9be01507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be01507f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400c820 tx=0x7f9bd400cce0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 shutdown_connections 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9be0158e30 0x7f9be00ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9be0158a50 0x7f9be014f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9bc406c8f0 0x7f9bc406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 --2- 172.21.15.135:0/1208914692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9be006f610 0x7f9be01507f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 >> 172.21.15.135:0/1208914692 conn(0x7f9be00d0520 msgr2=0x7f9be015a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 shutdown_connections 2024-04-17T06:10:24.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:24.997+0000 7f9be7235640 1 -- 172.21.15.135:0/1208914692 wait complete. 2024-04-17T06:10:25.006 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-17T06:10:25.109 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.111+0000 7f4e31eba640 1 Processor -- start 2024-04-17T06:10:25.110 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.111+0000 7f4e31eba640 1 -- start start 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.112+0000 7f4e31eba640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4e2c15bb00 0x7f4e2c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.112+0000 7f4e31eba640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e2c0b3a50 con 0x7f4e2c0b3590 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.112+0000 7f4e31eba640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e2c0b3d30 con 0x7f4e2c0afdd0 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.112+0000 7f4e31eba640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f4e2c0b4660 con 0x7f4e2c15bb00 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.112+0000 7f4e2b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4e2c15bb00 0x7f4e2c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.112+0000 7f4e2b7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f4e2c15bb00 0x7f4e2c0b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54596/0 (socket says 172.21.15.135:54596) 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.112+0000 7f4e2b7fe640 1 -- 172.21.15.135:0/4236785583 learned_addr learned my addr 172.21.15.135:0/4236785583 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.112+0000 7f4e2affd640 1 --1- 172.21.15.135:0/4236785583 >> v1:172.21.15.135:6789/0 conn(0x7f4e2c0b3590 0x7f4e2c0af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41812/0 (socket says 172.21.15.135:41812) 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.113+0000 7f4e2b7fe640 1 -- 172.21.15.135:0/4236785583 >> v1:172.21.15.190:6789/0 conn(0x7f4e2c0afdd0 legacy=0x7f4e2c0ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.113+0000 7f4e2b7fe640 1 -- 172.21.15.135:0/4236785583 >> v1:172.21.15.135:6789/0 conn(0x7f4e2c0b3590 legacy=0x7f4e2c0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.111 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.113+0000 7f4e2b7fe640 1 -- 172.21.15.135:0/4236785583 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c0b4010 con 0x7f4e2c15bb00 2024-04-17T06:10:25.112 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.113+0000 7f4e2a7fc640 1 -- 172.21.15.135:0/4236785583 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860614124 0 0) 0x7f4e2c0b3a50 con 0x7f4e2c0b3590 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.113+0000 7f4e2b7fe640 1 --2- 172.21.15.135:0/4236785583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c0b5150 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009e50 tx=0x7f4e1c02bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8b484d104457ed server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e2a7fc640 1 -- 172.21.15.135:0/4236785583 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c02cb30 con 0x7f4e2c15bb00 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e2a7fc640 1 -- 172.21.15.135:0/4236785583 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1c02ccd0 con 0x7f4e2c15bb00 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e2a7fc640 1 -- 172.21.15.135:0/4236785583 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c034610 con 0x7f4e2c15bb00 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e31eba640 1 -- 172.21.15.135:0/4236785583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 msgr2=0x7f4e2c0b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e31eba640 1 --2- 172.21.15.135:0/4236785583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c0b5150 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009e50 tx=0x7f4e1c02bf90 comp rx=0 tx=0).stop 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e31eba640 1 -- 172.21.15.135:0/4236785583 shutdown_connections 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e31eba640 1 --2- 172.21.15.135:0/4236785583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c0b5150 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.113 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e31eba640 1 -- 172.21.15.135:0/4236785583 >> 172.21.15.135:0/4236785583 conn(0x7f4e2c0d0520 msgr2=0x7f4e2c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:25.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e31eba640 1 -- 172.21.15.135:0/4236785583 shutdown_connections 2024-04-17T06:10:25.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.114+0000 7f4e31eba640 1 -- 172.21.15.135:0/4236785583 wait complete. 2024-04-17T06:10:25.114 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.115+0000 7f4e31eba640 1 Processor -- start 2024-04-17T06:10:25.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.115+0000 7f4e31eba640 1 -- start start 2024-04-17T06:10:25.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e31eba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c0afdd0 0x7f4e2c13f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e31eba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e2c0b3590 0x7f4e2c15e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e31eba640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e31eba640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c0b5810 con 0x7f4e2c15bb00 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e31eba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c0b5690 con 0x7f4e2c0afdd0 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e31eba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2c0cee30 con 0x7f4e2c0b3590 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c15e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54602/0 (socket says 172.21.15.135:54602) 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.116+0000 7f4e2bfff640 1 -- 172.21.15.135:0/2070850854 learned_addr learned my addr 172.21.15.135:0/2070850854 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2b7fe640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c0afdd0 0x7f4e2c13f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2affd640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e2c0b3590 0x7f4e2c15e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2bfff640 1 -- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e2c0b3590 msgr2=0x7f4e2c15e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2bfff640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e2c0b3590 0x7f4e2c15e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2bfff640 1 -- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c0afdd0 msgr2=0x7f4e2c13f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2bfff640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c0afdd0 0x7f4e2c13f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2bfff640 1 -- 172.21.15.135:0/2070850854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c13fa30 con 0x7f4e2c15bb00 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2b7fe640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c0afdd0 0x7f4e2c13f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:25.117 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2affd640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e2c0b3590 0x7f4e2c15e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:25.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e2bfff640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c15e6f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400c3e0 tx=0x7f4e2400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e13fff640 1 -- 172.21.15.135:0/2070850854 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24011790 con 0x7f4e2c15bb00 2024-04-17T06:10:25.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e13fff640 1 -- 172.21.15.135:0/2070850854 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e24017070 con 0x7f4e2c15bb00 2024-04-17T06:10:25.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1f6520 con 0x7f4e2c15bb00 2024-04-17T06:10:25.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e13fff640 1 -- 172.21.15.135:0/2070850854 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24011bb0 con 0x7f4e2c15bb00 2024-04-17T06:10:25.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.117+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c142240 con 0x7f4e2c15bb00 2024-04-17T06:10:25.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.118+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c074940 con 0x7f4e2c15bb00 2024-04-17T06:10:25.118 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.118+0000 7f4e13fff640 1 -- 172.21.15.135:0/2070850854 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e2401c460 con 0x7f4e2c15bb00 2024-04-17T06:10:25.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.119+0000 7f4e13fff640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4e1806c920 0x7f4e1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.119+0000 7f4e13fff640 1 -- 172.21.15.135:0/2070850854 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7f4e2408b720 con 0x7f4e2c15bb00 2024-04-17T06:10:25.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.119+0000 7f4e2b7fe640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4e1806c920 0x7f4e1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.119 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.119+0000 7f4e2b7fe640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4e1806c920 0x7f4e1806ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c035000 tx=0x7f4e1c030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.121 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.123+0000 7f4e13fff640 1 -- 172.21.15.135:0/2070850854 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e24059d10 con 0x7f4e2c15bb00 2024-04-17T06:10:25.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.302+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4e2c0723f0 con 0x7f4e2c15bb00 2024-04-17T06:10:25.300 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.302+0000 7f4e13fff640 1 -- 172.21.15.135:0/2070850854 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) ==== 84+0+65 (secure 0 0 0) 0x7f4e24059610 con 0x7f4e2c15bb00 2024-04-17T06:10:25.300 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:25.300 INFO:teuthology.orchestra.run.smithi135.stdout:[ 2024-04-17T06:10:25.301 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:25.301 INFO:teuthology.orchestra.run.smithi135.stdout: "poolnum": 1, 2024-04-17T06:10:25.301 INFO:teuthology.orchestra.run.smithi135.stdout: "poolname": ".mgr" 2024-04-17T06:10:25.301 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:25.301 INFO:teuthology.orchestra.run.smithi135.stdout:] 2024-04-17T06:10:25.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4e1806c920 msgr2=0x7f4e1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.302 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4e1806c920 0x7f4e1806ede0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c035000 tx=0x7f4e1c030000 comp rx=0 tx=0).stop 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 msgr2=0x7f4e2c15e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c15e6f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400c3e0 tx=0x7f4e2400c8a0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 shutdown_connections 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f4e1806c920 0x7f4e1806ede0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e2c15bb00 0x7f4e2c15e6f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e2c0b3590 0x7f4e2c15e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 --2- 172.21.15.135:0/2070850854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e2c0afdd0 0x7f4e2c13f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 >> 172.21.15.135:0/2070850854 conn(0x7f4e2c0d0520 msgr2=0x7f4e2c072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 shutdown_connections 2024-04-17T06:10:25.303 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.304+0000 7f4e31eba640 1 -- 172.21.15.135:0/2070850854 wait complete. 2024-04-17T06:10:25.313 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-17T06:10:25.313 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-17T06:10:25.420 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.422+0000 7fca5f97c640 1 Processor -- start 2024-04-17T06:10:25.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.423+0000 7fca5f97c640 1 -- start start 2024-04-17T06:10:25.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.423+0000 7fca5f97c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca5806d4b0 0x7fca5809b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.423+0000 7fca5f97c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca580b57c0 con 0x7fca5809b1e0 2024-04-17T06:10:25.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.423+0000 7fca5f97c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca5806da20 con 0x7fca58162450 2024-04-17T06:10:25.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.423+0000 7fca5f97c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fca580ccfc0 con 0x7fca5806d4b0 2024-04-17T06:10:25.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.423+0000 7fca5cf17640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fca5809b1e0 0x7fca580cdcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41814/0 (socket says 172.21.15.135:41814) 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.423+0000 7fca5cf17640 1 -- 172.21.15.135:0/2228026644 learned_addr learned my addr 172.21.15.135:0/2228026644 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca5df19640 1 --2- 172.21.15.135:0/2228026644 >> v2:172.21.15.190:3300/0 conn(0x7fca5806d4b0 0x7fca5809b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213228506 0 0) 0x7fca4c001110 con 0x7fca5809b1e0 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca40003520 con 0x7fca5809b1e0 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3629598772 0 0) 0x7fca40003520 con 0x7fca5809b1e0 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 >> v2:172.21.15.190:3300/0 conn(0x7fca5806d4b0 msgr2=0x7fca5809b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 --2- 172.21.15.135:0/2228026644 >> v2:172.21.15.190:3300/0 conn(0x7fca5806d4b0 0x7fca5809b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 >> v1:172.21.15.190:6789/0 conn(0x7fca58162450 legacy=0x7fca5809aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca580cd210 con 0x7fca5809b1e0 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7fca4c001840 con 0x7fca5809b1e0 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca5df19640 1 --2- 172.21.15.135:0/2228026644 >> v2:172.21.15.190:3300/0 conn(0x7fca5806d4b0 0x7fca5809b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 >> v1:172.21.15.135:6789/0 conn(0x7fca5809b1e0 legacy=0x7fca580cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca400039b0 0x7fca40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca40004800 0x7fca40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca580ccfc0 con 0x7fca400052e0 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca4000c1b0 con 0x7fca40004800 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca4000c3b0 con 0x7fca400039b0 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca5df19640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca5cf17640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca40004800 0x7fca40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca5d718640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca400039b0 0x7fca40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca5df19640 1 -- 172.21.15.135:0/2228026644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca400039b0 msgr2=0x7fca40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.424+0000 7fca5df19640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca400039b0 0x7fca40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.425+0000 7fca5df19640 1 -- 172.21.15.135:0/2228026644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca40004800 msgr2=0x7fca40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:25.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.425+0000 7fca5df19640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca40004800 0x7fca40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.425+0000 7fca5df19640 1 -- 172.21.15.135:0/2228026644 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca40003520 con 0x7fca400052e0 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.425+0000 7fca5df19640 1 -- 172.21.15.135:0/2228026644 reap_dead start 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.425+0000 7fca5df19640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca4000b9c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca50006fd0 tx=0x7fca40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba945878d82a4a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.425+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000f040 con 0x7fca400052e0 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.425+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca50004220 con 0x7fca400052e0 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.425+0000 7fca57fff640 1 -- 172.21.15.135:0/2228026644 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50006df0 con 0x7fca400052e0 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.426+0000 7fca5f97c640 1 -- 172.21.15.135:0/2228026644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 msgr2=0x7fca4000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.426+0000 7fca5f97c640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca4000b9c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca50006fd0 tx=0x7fca40005830 comp rx=0 tx=0).stop 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.426+0000 7fca5f97c640 1 -- 172.21.15.135:0/2228026644 shutdown_connections 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.426+0000 7fca5f97c640 1 --2- 172.21.15.135:0/2228026644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca4000b9c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.426+0000 7fca5f97c640 1 -- 172.21.15.135:0/2228026644 >> 172.21.15.135:0/2228026644 conn(0x7fca580b4460 msgr2=0x7fca580b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.426+0000 7fca5f97c640 1 -- 172.21.15.135:0/2228026644 shutdown_connections 2024-04-17T06:10:25.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.426+0000 7fca5f97c640 1 -- 172.21.15.135:0/2228026644 wait complete. 2024-04-17T06:10:25.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.427+0000 7fca5f97c640 1 Processor -- start 2024-04-17T06:10:25.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.427+0000 7fca5f97c640 1 -- start start 2024-04-17T06:10:25.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5f97c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca580caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5f97c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5815e320 0x7fca5815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5f97c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca580b59a0 0x7fca580cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5f97c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca5809bec0 con 0x7fca400052e0 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5f97c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fca580cd7d0 con 0x7fca5815e320 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5f97c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fca5809c040 con 0x7fca580b59a0 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5d718640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca580caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5d718640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca580caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54620/0 (socket says 172.21.15.135:54620) 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5d718640 1 -- 172.21.15.135:0/3527639991 learned_addr learned my addr 172.21.15.135:0/3527639991 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5cf17640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5815e320 0x7fca5815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5df19640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca580b59a0 0x7fca580cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5d718640 1 -- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca580b59a0 msgr2=0x7fca580cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5d718640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca580b59a0 0x7fca580cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5d718640 1 -- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5815e320 msgr2=0x7fca5815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5d718640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5815e320 0x7fca5815db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5d718640 1 -- 172.21.15.135:0/3527639991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca580cb4c0 con 0x7fca400052e0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.428+0000 7fca5df19640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca580b59a0 0x7fca580cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.429+0000 7fca5d718640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca580caf80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fca4800f9c0 tx=0x7fca4800fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.429+0000 7fca55ffb640 1 -- 172.21.15.135:0/3527639991 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48003b80 con 0x7fca400052e0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.429+0000 7fca55ffb640 1 -- 172.21.15.135:0/3527639991 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca48018070 con 0x7fca400052e0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.429+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca580c75e0 con 0x7fca400052e0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.429+0000 7fca55ffb640 1 -- 172.21.15.135:0/3527639991 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4800d850 con 0x7fca400052e0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.429+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca580c7ba0 con 0x7fca400052e0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.430+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca58068a20 con 0x7fca400052e0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.430+0000 7fca55ffb640 1 -- 172.21.15.135:0/3527639991 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca4800d9f0 con 0x7fca400052e0 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.431+0000 7fca55ffb640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca4406c8f0 0x7fca4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.431+0000 7fca55ffb640 1 -- 172.21.15.135:0/3527639991 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5249+0+0 (secure 0 0 0) 0x7fca4808ba00 con 0x7fca400052e0 2024-04-17T06:10:25.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.431+0000 7fca5cf17640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca4406c8f0 0x7fca4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.431+0000 7fca5cf17640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca4406c8f0 0x7fca4406edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca580c9240 tx=0x7fca4c001340 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.435+0000 7fca55ffb640 1 -- 172.21.15.135:0/3527639991 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca580cb4c0 con 0x7fca400052e0 2024-04-17T06:10:25.610 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.612+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fca580d1e20 con 0x7fca400052e0 2024-04-17T06:10:25.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.757+0000 7fca55ffb640 1 -- 172.21.15.135:0/3527639991 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) ==== 138+0+0 (secure 0 0 0) 0x7fca48059ff0 con 0x7fca400052e0 2024-04-17T06:10:25.755 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'cephfs_metadata' created 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.759+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca4406c8f0 msgr2=0x7fca4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.759+0000 7fca5f97c640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca4406c8f0 0x7fca4406edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca580c9240 tx=0x7fca4c001340 comp rx=0 tx=0).stop 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.759+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 msgr2=0x7fca580caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.759+0000 7fca5f97c640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca580caf80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fca4800f9c0 tx=0x7fca4800fe80 comp rx=0 tx=0).stop 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.760+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 shutdown_connections 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.760+0000 7fca5f97c640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca580b59a0 0x7fca580cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.760+0000 7fca5f97c640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5815e320 0x7fca5815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.760+0000 7fca5f97c640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fca4406c8f0 0x7fca4406edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.760+0000 7fca5f97c640 1 --2- 172.21.15.135:0/3527639991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca400052e0 0x7fca580caf80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.760+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 >> 172.21.15.135:0/3527639991 conn(0x7fca580b4460 msgr2=0x7fca58070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:25.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.760+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 shutdown_connections 2024-04-17T06:10:25.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.760+0000 7fca5f97c640 1 -- 172.21.15.135:0/3527639991 wait complete. 2024-04-17T06:10:25.771 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-17T06:10:25.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.882+0000 7f55a9f3d640 1 Processor -- start 2024-04-17T06:10:25.881 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.882+0000 7f55a9f3d640 1 -- start start 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a9f3d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55a415e9c0 0x7f55a40ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a9f3d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f55a40d2ce0 0x7f55a40cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a9f3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a40cfb20 0x7f55a40730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a9f3d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f55a40cff70 con 0x7f55a415e9c0 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a9f3d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f55a40d3150 con 0x7f55a40d2ce0 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a9f3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f55a40d33e0 con 0x7f55a40cfb20 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55a415e9c0 0x7f55a40ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a37fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f55a415e9c0 0x7f55a40ba3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54630/0 (socket says 172.21.15.135:54630) 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.883+0000 7f55a37fe640 1 -- 172.21.15.135:0/1369794463 learned_addr learned my addr 172.21.15.135:0/1369794463 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:25.883 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.884+0000 7f55a37fe640 1 -- 172.21.15.135:0/1369794463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a40cfb20 msgr2=0x7f55a40730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.884+0000 7f55a37fe640 1 --2- 172.21.15.135:0/1369794463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a40cfb20 0x7f55a40730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.884+0000 7f55a37fe640 1 -- 172.21.15.135:0/1369794463 >> v2:172.21.15.190:3300/0 conn(0x7f55a40d2ce0 msgr2=0x7f55a40cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.884+0000 7f55a2ffd640 1 --2- 172.21.15.135:0/1369794463 >> v2:172.21.15.190:3300/0 conn(0x7f55a40d2ce0 0x7f55a40cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.884+0000 7f55a37fe640 1 --2- 172.21.15.135:0/1369794463 >> v2:172.21.15.190:3300/0 conn(0x7f55a40d2ce0 0x7f55a40cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.884+0000 7f55a37fe640 1 -- 172.21.15.135:0/1369794463 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a40d36c0 con 0x7f55a415e9c0 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.884+0000 7f55a37fe640 1 --2- 172.21.15.135:0/1369794463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a415e9c0 0x7f55a40ba3e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f55a408ce80 tx=0x7f559402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915f5dfd52ac4470 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a27fc640 1 -- 172.21.15.135:0/1369794463 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594034070 con 0x7f55a415e9c0 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a27fc640 1 -- 172.21.15.135:0/1369794463 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5594030860 con 0x7f55a415e9c0 2024-04-17T06:10:25.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a27fc640 1 -- 172.21.15.135:0/1369794463 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594030c30 con 0x7f55a415e9c0 2024-04-17T06:10:25.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 -- 172.21.15.135:0/1369794463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a415e9c0 msgr2=0x7f55a40ba3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/1369794463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a415e9c0 0x7f55a40ba3e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f55a408ce80 tx=0x7f559402b9a0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 -- 172.21.15.135:0/1369794463 shutdown_connections 2024-04-17T06:10:25.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/1369794463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a40cfb20 0x7f55a40730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/1369794463 >> v2:172.21.15.190:3300/0 conn(0x7f55a40d2ce0 0x7f55a40cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/1369794463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a415e9c0 0x7f55a40ba3e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 -- 172.21.15.135:0/1369794463 >> 172.21.15.135:0/1369794463 conn(0x7f55a406cc90 msgr2=0x7f55a406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:25.888 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 -- 172.21.15.135:0/1369794463 shutdown_connections 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.885+0000 7f55a9f3d640 1 -- 172.21.15.135:0/1369794463 wait complete. 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.886+0000 7f55a9f3d640 1 Processor -- start 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a9f3d640 1 -- start start 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a9f3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a40cfb20 0x7f55a41f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a9f3d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40d2ce0 0x7f55a415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a9f3d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a415e9c0 0x7f55a415f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a9f3d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40d3f60 con 0x7f55a40cfb20 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a9f3d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f55a40d3c80 con 0x7f55a40d2ce0 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a9f3d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f55a415eda0 con 0x7f55a415e9c0 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a40cfb20 0x7f55a41f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.887+0000 7f55a37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a40cfb20 0x7f55a41f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54640/0 (socket says 172.21.15.135:54640) 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a37fe640 1 -- 172.21.15.135:0/2768689582 learned_addr learned my addr 172.21.15.135:0/2768689582 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a2ffd640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40d2ce0 0x7f55a415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a3fff640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a415e9c0 0x7f55a415f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.889 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a37fe640 1 -- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a415e9c0 msgr2=0x7f55a415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a37fe640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a415e9c0 0x7f55a415f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a37fe640 1 -- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40d2ce0 msgr2=0x7f55a415d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a37fe640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40d2ce0 0x7f55a415d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a37fe640 1 -- 172.21.15.135:0/2768689582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a414b860 con 0x7f55a40cfb20 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a2ffd640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40d2ce0 0x7f55a415d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a3fff640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a415e9c0 0x7f55a415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a37fe640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a40cfb20 0x7f55a41f4650 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5594000c00 tx=0x7f5594004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f558bfff640 1 -- 172.21.15.135:0/2768689582 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594048070 con 0x7f55a40cfb20 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f558bfff640 1 -- 172.21.15.135:0/2768689582 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5594039d90 con 0x7f55a40cfb20 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f558bfff640 1 -- 172.21.15.135:0/2768689582 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594034040 con 0x7f55a40cfb20 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a414bb40 con 0x7f55a40cfb20 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.888+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41f4b90 con 0x7f55a40cfb20 2024-04-17T06:10:25.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.889+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a414c060 con 0x7f55a40cfb20 2024-04-17T06:10:25.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.890+0000 7f558bfff640 1 -- 172.21.15.135:0/2768689582 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f559404b070 con 0x7f55a40cfb20 2024-04-17T06:10:25.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.890+0000 7f558bfff640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f559006c8f0 0x7f559006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:25.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.890+0000 7f558bfff640 1 -- 172.21.15.135:0/2768689582 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5619+0+0 (secure 0 0 0) 0x7f55940adbc0 con 0x7f55a40cfb20 2024-04-17T06:10:25.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.890+0000 7f55a2ffd640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f559006c8f0 0x7f559006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:25.891 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.891+0000 7f55a2ffd640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f559006c8f0 0x7f559006edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55a40cee80 tx=0x7f55980074a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:25.893 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:25.895+0000 7f558bfff640 1 -- 172.21.15.135:0/2768689582 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f559407c0d0 con 0x7f55a40cfb20 2024-04-17T06:10:26.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.073+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f55a4074940 con 0x7f55a40cfb20 2024-04-17T06:10:26.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.764+0000 7f558bfff640 1 -- 172.21.15.135:0/2768689582 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) ==== 170+0+0 (secure 0 0 0) 0x7f559407b9d0 con 0x7f55a40cfb20 2024-04-17T06:10:26.763 INFO:teuthology.orchestra.run.smithi135.stderr:pool 'cephfs_data' created 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f559006c8f0 msgr2=0x7f559006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f559006c8f0 0x7f559006edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55a40cee80 tx=0x7f55980074a0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a40cfb20 msgr2=0x7f55a41f4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a40cfb20 0x7f55a41f4650 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5594000c00 tx=0x7f5594004730 comp rx=0 tx=0).stop 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 shutdown_connections 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55a415e9c0 0x7f55a415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55a40d2ce0 0x7f55a415d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f559006c8f0 0x7f559006edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 --2- 172.21.15.135:0/2768689582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55a40cfb20 0x7f55a41f4650 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 >> 172.21.15.135:0/2768689582 conn(0x7f55a406cc90 msgr2=0x7f55a40b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 shutdown_connections 2024-04-17T06:10:26.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.766+0000 7f55a9f3d640 1 -- 172.21.15.135:0/2768689582 wait complete. 2024-04-17T06:10:26.785 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-17T06:10:26.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.896+0000 7fcaf180d640 1 Processor -- start 2024-04-17T06:10:26.894 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.896+0000 7fcaf180d640 1 -- start start 2024-04-17T06:10:26.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaf180d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcaec15ec00 0x7fcaec0d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:26.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaf180d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcaec0d19e0 0x7fcaec0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaf180d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaec0d3330 con 0x7fcaec0d43d0 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaf180d640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec15efe0 con 0x7fcaec0d19e0 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaf180d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec0d3f40 con 0x7fcaec15ec00 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaeb7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcaec0d19e0 0x7fcaec0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaeb7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcaec0d19e0 0x7fcaec0b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54654/0 (socket says 172.21.15.135:54654) 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaeb7fe640 1 -- 172.21.15.135:0/1854850540 learned_addr learned my addr 172.21.15.135:0/1854850540 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaeaffd640 1 --2- 172.21.15.135:0/1854850540 >> v2:172.21.15.190:3300/0 conn(0x7fcaec15ec00 0x7fcaec0d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaeb7fe640 1 -- 172.21.15.135:0/1854850540 >> v2:172.21.15.190:3300/0 conn(0x7fcaec15ec00 msgr2=0x7fcaec0d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaeb7fe640 1 --2- 172.21.15.135:0/1854850540 >> v2:172.21.15.190:3300/0 conn(0x7fcaec15ec00 0x7fcaec0d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaeb7fe640 1 -- 172.21.15.135:0/1854850540 >> v1:172.21.15.190:6789/0 conn(0x7fcaec0d43d0 legacy=0x7fcaec0d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.897+0000 7fcaeb7fe640 1 -- 172.21.15.135:0/1854850540 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec0d3610 con 0x7fcaec0d19e0 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.898+0000 7fcaeb7fe640 1 --2- 172.21.15.135:0/1854850540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec0d19e0 0x7fcaec0b7a40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcaec06db40 tx=0x7fcae402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1206f17e4395de36 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:26.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.898+0000 7fcae9ffb640 1 -- 172.21.15.135:0/1854850540 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae402f850 con 0x7fcaec0d19e0 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.898+0000 7fcae9ffb640 1 -- 172.21.15.135:0/1854850540 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae402f9f0 con 0x7fcaec0d19e0 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.898+0000 7fcae9ffb640 1 -- 172.21.15.135:0/1854850540 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae402fdc0 con 0x7fcaec0d19e0 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.899+0000 7fcaf180d640 1 -- 172.21.15.135:0/1854850540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec0d19e0 msgr2=0x7fcaec0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.899+0000 7fcaf180d640 1 --2- 172.21.15.135:0/1854850540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec0d19e0 0x7fcaec0b7a40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcaec06db40 tx=0x7fcae402ba30 comp rx=0 tx=0).stop 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.899+0000 7fcaf180d640 1 -- 172.21.15.135:0/1854850540 shutdown_connections 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.899+0000 7fcaf180d640 1 --2- 172.21.15.135:0/1854850540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec0d19e0 0x7fcaec0b7a40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.899+0000 7fcaf180d640 1 --2- 172.21.15.135:0/1854850540 >> v2:172.21.15.190:3300/0 conn(0x7fcaec15ec00 0x7fcaec0d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.899+0000 7fcaf180d640 1 -- 172.21.15.135:0/1854850540 >> 172.21.15.135:0/1854850540 conn(0x7fcaec06cc90 msgr2=0x7fcaec06d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.899+0000 7fcaf180d640 1 -- 172.21.15.135:0/1854850540 shutdown_connections 2024-04-17T06:10:26.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.899+0000 7fcaf180d640 1 -- 172.21.15.135:0/1854850540 wait complete. 2024-04-17T06:10:26.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.900+0000 7fcaf180d640 1 Processor -- start 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.900+0000 7fcaf180d640 1 -- start start 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaf180d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaec0d19e0 0x7fcaec154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaf180d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaec0d43d0 0x7fcaec0c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaf180d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec15ec00 0x7fcaec15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaf180d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0d2ed0 con 0x7fcaec15ec00 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaf180d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0d3bd0 con 0x7fcaec0d43d0 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaf180d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec0d3050 con 0x7fcaec0d19e0 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec15ec00 0x7fcaec15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec15ec00 0x7fcaec15a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54656/0 (socket says 172.21.15.135:54656) 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeb7fe640 1 -- 172.21.15.135:0/214498126 learned_addr learned my addr 172.21.15.135:0/214498126 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaea7fc640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaec0d43d0 0x7fcaec0c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeaffd640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaec0d19e0 0x7fcaec154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:26.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeb7fe640 1 -- 172.21.15.135:0/214498126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaec0d19e0 msgr2=0x7fcaec154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeb7fe640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaec0d19e0 0x7fcaec154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeb7fe640 1 -- 172.21.15.135:0/214498126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaec0d43d0 msgr2=0x7fcaec0c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeb7fe640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaec0d43d0 0x7fcaec0c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.901+0000 7fcaeb7fe640 1 -- 172.21.15.135:0/214498126 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec0c8db0 con 0x7fcaec15ec00 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.902+0000 7fcaeb7fe640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec15ec00 0x7fcaec15a400 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcae40090b0 tx=0x7fcae4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.902+0000 7fcad3fff640 1 -- 172.21.15.135:0/214498126 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4037bc0 con 0x7fcaec15ec00 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.902+0000 7fcad3fff640 1 -- 172.21.15.135:0/214498126 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae4046070 con 0x7fcaec15ec00 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.902+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec0c9090 con 0x7fcaec15ec00 2024-04-17T06:10:26.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.902+0000 7fcad3fff640 1 -- 172.21.15.135:0/214498126 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4041400 con 0x7fcaec15ec00 2024-04-17T06:10:26.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.902+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec0c9650 con 0x7fcaec15ec00 2024-04-17T06:10:26.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.903+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec0ba4d0 con 0x7fcaec15ec00 2024-04-17T06:10:26.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.903+0000 7fcad3fff640 1 -- 172.21.15.135:0/214498126 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcae4040070 con 0x7fcaec15ec00 2024-04-17T06:10:26.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.904+0000 7fcad3fff640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcad806c8f0 0x7fcad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:26.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.904+0000 7fcad3fff640 1 -- 172.21.15.135:0/214498126 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6001+0+0 (secure 0 0 0) 0x7fcae40ae3f0 con 0x7fcaec15ec00 2024-04-17T06:10:26.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.904+0000 7fcaeaffd640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcad806c8f0 0x7fcad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:26.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.904+0000 7fcaeaffd640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcad806c8f0 0x7fcad806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0099c0 tx=0x7fcadc003990 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:26.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:26.908+0000 7fcad3fff640 1 -- 172.21.15.135:0/214498126 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcaec0ba4d0 con 0x7fcaec15ec00 2024-04-17T06:10:27.116 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.118+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcaec0cb790 con 0x7fcaec15ec00 2024-04-17T06:10:27.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.765+0000 7fcad3fff640 1 -- 172.21.15.135:0/214498126 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fcaec0cb790 con 0x7fcaec15ec00 2024-04-17T06:10:27.764 INFO:teuthology.orchestra.run.smithi135.stderr:new fs with metadata pool 5 and data pool 6 2024-04-17T06:10:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcad806c8f0 msgr2=0x7fcad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcad806c8f0 0x7fcad806edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0099c0 tx=0x7fcadc003990 comp rx=0 tx=0).stop 2024-04-17T06:10:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec15ec00 msgr2=0x7fcaec15a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec15ec00 0x7fcaec15a400 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcae40090b0 tx=0x7fcae4004730 comp rx=0 tx=0).stop 2024-04-17T06:10:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 shutdown_connections 2024-04-17T06:10:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcad806c8f0 0x7fcad806edb0 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0099c0 tx=0x7fcadc003990 comp rx=0 tx=0).stop 2024-04-17T06:10:27.766 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaec15ec00 0x7fcaec15a400 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:27.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaec0d43d0 0x7fcaec0c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:27.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 --2- 172.21.15.135:0/214498126 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaec0d19e0 0x7fcaec154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:27.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 >> 172.21.15.135:0/214498126 conn(0x7fcaec06cc90 msgr2=0x7fcaec0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:27.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 shutdown_connections 2024-04-17T06:10:27.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.768+0000 7fcaf180d640 1 -- 172.21.15.135:0/214498126 wait complete. 2024-04-17T06:10:27.787 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:27.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.897+0000 7f67ced46640 1 Processor -- start 2024-04-17T06:10:27.895 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.897+0000 7f67ced46640 1 -- start start 2024-04-17T06:10:27.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ced46640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67c815c8a0 0x7f67c80b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:27.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ced46640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c806ba30 con 0x7f67c80b3d40 2024-04-17T06:10:27.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ced46640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c806bcd0 con 0x7f67c80cd7b0 2024-04-17T06:10:27.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ced46640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f67c806c600 con 0x7f67c815c8a0 2024-04-17T06:10:27.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ccae2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67c815c8a0 0x7f67c80b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:27.897 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ccae2640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f67c815c8a0 0x7f67c80b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54668/0 (socket says 172.21.15.135:54668) 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ccae2640 1 -- 172.21.15.135:0/2852746917 learned_addr learned my addr 172.21.15.135:0/2852746917 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67bffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f67c80b3d40 0x7f67c80cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41828/0 (socket says 172.21.15.135:41828) 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ccae2640 1 -- 172.21.15.135:0/2852746917 >> v1:172.21.15.190:6789/0 conn(0x7f67c80cd7b0 legacy=0x7f67c815cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ccae2640 1 -- 172.21.15.135:0/2852746917 >> v1:172.21.15.135:6789/0 conn(0x7f67c80b3d40 legacy=0x7f67c80cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.898+0000 7f67ccae2640 1 -- 172.21.15.135:0/2852746917 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c806bfb0 con 0x7f67c815c8a0 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.899+0000 7f67ccae2640 1 --2- 172.21.15.135:0/2852746917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c815c8a0 0x7f67c80b3800 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009870 tx=0x7f67c402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5590aad3433cfa01 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.899+0000 7f67bf7fe640 1 -- 172.21.15.135:0/2852746917 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4034070 con 0x7f67c815c8a0 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.899+0000 7f67bf7fe640 1 -- 172.21.15.135:0/2852746917 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c4030860 con 0x7f67c815c8a0 2024-04-17T06:10:27.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.899+0000 7f67bf7fe640 1 -- 172.21.15.135:0/2852746917 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4030c30 con 0x7f67c815c8a0 2024-04-17T06:10:27.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.900+0000 7f67ced46640 1 -- 172.21.15.135:0/2852746917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c815c8a0 msgr2=0x7f67c80b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:27.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.900+0000 7f67ced46640 1 --2- 172.21.15.135:0/2852746917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c815c8a0 0x7f67c80b3800 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009870 tx=0x7f67c402b9a0 comp rx=0 tx=0).stop 2024-04-17T06:10:27.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.900+0000 7f67ced46640 1 -- 172.21.15.135:0/2852746917 shutdown_connections 2024-04-17T06:10:27.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.900+0000 7f67ced46640 1 --2- 172.21.15.135:0/2852746917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c815c8a0 0x7f67c80b3800 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.900+0000 7f67ced46640 1 -- 172.21.15.135:0/2852746917 >> 172.21.15.135:0/2852746917 conn(0x7f67c809a9f0 msgr2=0x7f67c80b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.900+0000 7f67ced46640 1 -- 172.21.15.135:0/2852746917 shutdown_connections 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.900+0000 7f67ced46640 1 -- 172.21.15.135:0/2852746917 wait complete. 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.901+0000 7f67ced46640 1 Processor -- start 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.901+0000 7f67ced46640 1 -- start start 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67ced46640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80b3d40 0x7f67c80c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67ced46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c80cd7b0 0x7f67c80c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67ced46640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67c815c8a0 0x7f67c80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67ced46640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67c80d0370 con 0x7f67c80cd7b0 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67ced46640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f67c80ced60 con 0x7f67c80b3d40 2024-04-17T06:10:27.902 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67ced46640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f67c80d04f0 con 0x7f67c815c8a0 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c80cd7b0 0x7f67c80c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c80cd7b0 0x7f67c80c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54676/0 (socket says 172.21.15.135:54676) 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 -- 172.21.15.135:0/1679757856 learned_addr learned my addr 172.21.15.135:0/1679757856 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 -- 172.21.15.135:0/1679757856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67c815c8a0 msgr2=0x7f67c80b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67c815c8a0 0x7f67c80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 -- 172.21.15.135:0/1679757856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80b3d40 msgr2=0x7f67c80c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67ccae2640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80b3d40 0x7f67c80c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80b3d40 0x7f67c80c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 -- 172.21.15.135:0/1679757856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c81f5690 con 0x7f67c80cd7b0 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bffff640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c80cd7b0 0x7f67c80c5c60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c430 tx=0x7f67b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.902+0000 7f67bd7fa640 1 -- 172.21.15.135:0/1679757856 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b80074e0 con 0x7f67c80cd7b0 2024-04-17T06:10:27.903 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.903+0000 7f67bd7fa640 1 -- 172.21.15.135:0/1679757856 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b8007680 con 0x7f67c80cd7b0 2024-04-17T06:10:27.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.903+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c81f53b0 con 0x7f67c80cd7b0 2024-04-17T06:10:27.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.903+0000 7f67bd7fa640 1 -- 172.21.15.135:0/1679757856 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8004a20 con 0x7f67c80cd7b0 2024-04-17T06:10:27.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.903+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c80c5460 con 0x7f67c80cd7b0 2024-04-17T06:10:27.904 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.903+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c8074940 con 0x7f67c80cd7b0 2024-04-17T06:10:27.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.904+0000 7f67bd7fa640 1 -- 172.21.15.135:0/1679757856 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67b801a020 con 0x7f67c80cd7b0 2024-04-17T06:10:27.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.904+0000 7f67bd7fa640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f67b406c8f0 0x7f67b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:27.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.905+0000 7f67bd7fa640 1 -- 172.21.15.135:0/1679757856 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7f67b808b4a0 con 0x7f67c80cd7b0 2024-04-17T06:10:27.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.905+0000 7f67ccae2640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f67b406c8f0 0x7f67b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:27.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.905+0000 7f67ccae2640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f67b406c8f0 0x7f67b406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67c4036000 tx=0x7f67c403b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:27.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:27.909+0000 7f67bd7fa640 1 -- 172.21.15.135:0/1679757856 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67b80597d0 con 0x7f67c80cd7b0 2024-04-17T06:10:28.084 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.086+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67c815a8c0 con 0x7f67c80cd7b0 2024-04-17T06:10:28.085 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.087+0000 7f67bd7fa640 1 -- 172.21.15.135:0/1679757856 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+17003 (secure 0 0 0) 0x7f67b80590d0 con 0x7f67c80cd7b0 2024-04-17T06:10:28.085 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:28.086 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":37,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:27.119832+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:10:25.616330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-17T06:10:26.077318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:28.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f67b406c8f0 msgr2=0x7f67b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.087 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f67b406c8f0 0x7f67b406edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67c4036000 tx=0x7f67c403b040 comp rx=0 tx=0).stop 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c80cd7b0 msgr2=0x7f67c80c5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c80cd7b0 0x7f67c80c5c60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f67b800c430 tx=0x7f67b800c8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 shutdown_connections 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67c815c8a0 0x7f67c80b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f67b406c8f0 0x7f67b406edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67c80cd7b0 0x7f67c80c5c60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 --2- 172.21.15.135:0/1679757856 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67c80b3d40 0x7f67c80c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 >> 172.21.15.135:0/1679757856 conn(0x7f67c809a9f0 msgr2=0x7f67c815ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 shutdown_connections 2024-04-17T06:10:28.088 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.089+0000 7f67ced46640 1 -- 172.21.15.135:0/1679757856 wait complete. 2024-04-17T06:10:28.099 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:28.202 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.204+0000 7fbd9dd89640 1 Processor -- start 2024-04-17T06:10:28.203 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.204+0000 7fbd9dd89640 1 -- start start 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd9dd89640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbd9815fbe0 0x7fbd980d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd9dd89640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbd980d0d50 0x7fbd980cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd9dd89640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd98070e60 con 0x7fbd9815f800 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd9dd89640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9806c8f0 con 0x7fbd9815fbe0 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd9dd89640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9806f070 con 0x7fbd980d0d50 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd96ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbd9815fbe0 0x7fbd980d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd96ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fbd9815fbe0 0x7fbd980d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54684/0 (socket says 172.21.15.135:54684) 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd96ffd640 1 -- 172.21.15.135:0/2125971934 learned_addr learned my addr 172.21.15.135:0/2125971934 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.205+0000 7fbd97fff640 1 --2- 172.21.15.135:0/2125971934 >> v2:172.21.15.190:3300/0 conn(0x7fbd980d0d50 0x7fbd980cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.204 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.206+0000 7fbd96ffd640 1 -- 172.21.15.135:0/2125971934 >> v2:172.21.15.190:3300/0 conn(0x7fbd980d0d50 msgr2=0x7fbd980cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:28.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.206+0000 7fbd96ffd640 1 --2- 172.21.15.135:0/2125971934 >> v2:172.21.15.190:3300/0 conn(0x7fbd980d0d50 0x7fbd980cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.206+0000 7fbd96ffd640 1 -- 172.21.15.135:0/2125971934 >> v1:172.21.15.190:6789/0 conn(0x7fbd9815f800 legacy=0x7fbd98070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.205 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.206+0000 7fbd96ffd640 1 -- 172.21.15.135:0/2125971934 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9806bb30 con 0x7fbd9815fbe0 2024-04-17T06:10:28.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.206+0000 7fbd96ffd640 1 --2- 172.21.15.135:0/2125971934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd9815fbe0 0x7fbd980d0810 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbd9809afa0 tx=0x7fbd8c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3308ce47421aeed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.206+0000 7fbd967fc640 1 -- 172.21.15.135:0/2125971934 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c02f850 con 0x7fbd9815fbe0 2024-04-17T06:10:28.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.206+0000 7fbd967fc640 1 -- 172.21.15.135:0/2125971934 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8c02f9f0 con 0x7fbd9815fbe0 2024-04-17T06:10:28.206 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.206+0000 7fbd967fc640 1 -- 172.21.15.135:0/2125971934 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c02fdc0 con 0x7fbd9815fbe0 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.207+0000 7fbd9dd89640 1 -- 172.21.15.135:0/2125971934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd9815fbe0 msgr2=0x7fbd980d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.207+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/2125971934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd9815fbe0 0x7fbd980d0810 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbd9809afa0 tx=0x7fbd8c02ba30 comp rx=0 tx=0).stop 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.207+0000 7fbd9dd89640 1 -- 172.21.15.135:0/2125971934 shutdown_connections 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.207+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/2125971934 >> v2:172.21.15.190:3300/0 conn(0x7fbd980d0d50 0x7fbd980cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.207+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/2125971934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd9815fbe0 0x7fbd980d0810 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.207+0000 7fbd9dd89640 1 -- 172.21.15.135:0/2125971934 >> 172.21.15.135:0/2125971934 conn(0x7fbd9809a9f0 msgr2=0x7fbd980b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.207+0000 7fbd9dd89640 1 -- 172.21.15.135:0/2125971934 shutdown_connections 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.207+0000 7fbd9dd89640 1 -- 172.21.15.135:0/2125971934 wait complete. 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.208+0000 7fbd9dd89640 1 Processor -- start 2024-04-17T06:10:28.207 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.208+0000 7fbd9dd89640 1 -- start start 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd9dd89640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd980d0d50 0x7fbd98153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd9dd89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd9815f800 0x7fbd981f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd9dd89640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd9815fbe0 0x7fbd98152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd9dd89640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9806c0f0 con 0x7fbd980d0d50 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd9dd89640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9806c460 con 0x7fbd9815fbe0 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd9dd89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fbd980ce340 con 0x7fbd9815f800 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd977fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd980d0d50 0x7fbd98153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd977fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd980d0d50 0x7fbd98153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54692/0 (socket says 172.21.15.135:54692) 2024-04-17T06:10:28.210 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd977fe640 1 -- 172.21.15.135:0/1934564843 learned_addr learned my addr 172.21.15.135:0/1934564843 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd96ffd640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd9815f800 0x7fbd981f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.209+0000 7fbd97fff640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd9815fbe0 0x7fbd98152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd977fe640 1 -- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd9815f800 msgr2=0x7fbd981f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd977fe640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd9815f800 0x7fbd981f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd977fe640 1 -- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd9815fbe0 msgr2=0x7fbd98152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd977fe640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd9815fbe0 0x7fbd98152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd977fe640 1 -- 172.21.15.135:0/1934564843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd980b6cb0 con 0x7fbd980d0d50 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd97fff640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd9815fbe0 0x7fbd98152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd977fe640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd980d0d50 0x7fbd98153c80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c790 tx=0x7fbd8800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd7ffff640 1 -- 172.21.15.135:0/1934564843 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88016020 con 0x7fbd980d0d50 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd7ffff640 1 -- 172.21.15.135:0/1934564843 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd88006d50 con 0x7fbd980d0d50 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd980b6f90 con 0x7fbd980d0d50 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd7ffff640 1 -- 172.21.15.135:0/1934564843 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88003bb0 con 0x7fbd980d0d50 2024-04-17T06:10:28.211 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.210+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981537d0 con 0x7fbd980d0d50 2024-04-17T06:10:28.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.211+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd981532e0 con 0x7fbd980d0d50 2024-04-17T06:10:28.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.212+0000 7fbd7ffff640 1 -- 172.21.15.135:0/1934564843 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd8800a7f0 con 0x7fbd980d0d50 2024-04-17T06:10:28.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.212+0000 7fbd7ffff640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbd8406c920 0x7fbd8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.212+0000 7fbd96ffd640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbd8406c920 0x7fbd8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.212+0000 7fbd7ffff640 1 -- 172.21.15.135:0/1934564843 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7fbd8808b880 con 0x7fbd980d0d50 2024-04-17T06:10:28.212 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.212+0000 7fbd96ffd640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbd8406c920 0x7fbd8406ede0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c036000 tx=0x7fbd8c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.214 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.216+0000 7fbd7ffff640 1 -- 172.21.15.135:0/1934564843 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd88020020 con 0x7fbd980d0d50 2024-04-17T06:10:28.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.392+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd98074940 con 0x7fbd980d0d50 2024-04-17T06:10:28.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.393+0000 7fbd7ffff640 1 -- 172.21.15.135:0/1934564843 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+17003 (secure 0 0 0) 0x7fbd88059bb0 con 0x7fbd980d0d50 2024-04-17T06:10:28.391 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:28.392 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":37,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:27.119832+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:10:25.616330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-17T06:10:26.077318+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:28.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbd8406c920 msgr2=0x7fbd8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbd8406c920 0x7fbd8406ede0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c036000 tx=0x7fbd8c008030 comp rx=0 tx=0).stop 2024-04-17T06:10:28.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd980d0d50 msgr2=0x7fbd98153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd980d0d50 0x7fbd98153c80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c790 tx=0x7fbd8800cc50 comp rx=0 tx=0).stop 2024-04-17T06:10:28.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 shutdown_connections 2024-04-17T06:10:28.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbd9815fbe0 0x7fbd98152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbd9815f800 0x7fbd981f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fbd8406c920 0x7fbd8406ede0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 --2- 172.21.15.135:0/1934564843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd980d0d50 0x7fbd98153c80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 >> 172.21.15.135:0/1934564843 conn(0x7fbd9809a9f0 msgr2=0x7fbd980ae600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:28.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 shutdown_connections 2024-04-17T06:10:28.395 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.395+0000 7fbd9dd89640 1 -- 172.21.15.135:0/1934564843 wait complete. 2024-04-17T06:10:28.405 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-17T06:10:28.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.513+0000 7ff5b782a640 1 Processor -- start 2024-04-17T06:10:28.512 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.514+0000 7ff5b782a640 1 -- start start 2024-04-17T06:10:28.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b782a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5b015ec00 0x7ff5b00d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b782a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff5b00d19e0 0x7ff5b00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b782a640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5b00d3330 con 0x7ff5b00d43d0 2024-04-17T06:10:28.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b782a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b015efe0 con 0x7ff5b015ec00 2024-04-17T06:10:28.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b782a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b00d3f40 con 0x7ff5b00d19e0 2024-04-17T06:10:28.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b55c6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5b015ec00 0x7ff5b00d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b55c6640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7ff5b015ec00 0x7ff5b00d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54706/0 (socket says 172.21.15.135:54706) 2024-04-17T06:10:28.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b4dc5640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7ff5b00d43d0 0x7ff5b00d12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41840/0 (socket says 172.21.15.135:41840) 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b55c6640 1 -- 172.21.15.135:0/1297159143 learned_addr learned my addr 172.21.15.135:0/1297159143 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b5dc7640 1 --2- 172.21.15.135:0/1297159143 >> v2:172.21.15.190:3300/0 conn(0x7ff5b00d19e0 0x7ff5b00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b55c6640 1 -- 172.21.15.135:0/1297159143 >> v2:172.21.15.190:3300/0 conn(0x7ff5b00d19e0 msgr2=0x7ff5b00b7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b55c6640 1 --2- 172.21.15.135:0/1297159143 >> v2:172.21.15.190:3300/0 conn(0x7ff5b00d19e0 0x7ff5b00b7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b55c6640 1 -- 172.21.15.135:0/1297159143 >> v1:172.21.15.135:6789/0 conn(0x7ff5b00d43d0 legacy=0x7ff5b00d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5b55c6640 1 -- 172.21.15.135:0/1297159143 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b00d3610 con 0x7ff5b015ec00 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.515+0000 7ff5a7fff640 1 -- 172.21.15.135:0/1297159143 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3047455360 0 0) 0x7ff5b00d3330 con 0x7ff5b00d43d0 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5b55c6640 1 --2- 172.21.15.135:0/1297159143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b015ec00 0x7ff5b00d2990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff5b006dab0 tx=0x7ff5ac02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bceb09dee219e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5a7fff640 1 -- 172.21.15.135:0/1297159143 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac02f850 con 0x7ff5b015ec00 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5a7fff640 1 -- 172.21.15.135:0/1297159143 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac02f9f0 con 0x7ff5b015ec00 2024-04-17T06:10:28.515 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5a7fff640 1 -- 172.21.15.135:0/1297159143 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac02fdc0 con 0x7ff5b015ec00 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5b782a640 1 -- 172.21.15.135:0/1297159143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b015ec00 msgr2=0x7ff5b00d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1297159143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b015ec00 0x7ff5b00d2990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff5b006dab0 tx=0x7ff5ac02ba30 comp rx=0 tx=0).stop 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5b782a640 1 -- 172.21.15.135:0/1297159143 shutdown_connections 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1297159143 >> v2:172.21.15.190:3300/0 conn(0x7ff5b00d19e0 0x7ff5b00b7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1297159143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b015ec00 0x7ff5b00d2990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.516+0000 7ff5b782a640 1 -- 172.21.15.135:0/1297159143 >> 172.21.15.135:0/1297159143 conn(0x7ff5b006cc90 msgr2=0x7ff5b006d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.517+0000 7ff5b782a640 1 -- 172.21.15.135:0/1297159143 shutdown_connections 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.517+0000 7ff5b782a640 1 -- 172.21.15.135:0/1297159143 wait complete. 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.518+0000 7ff5b782a640 1 Processor -- start 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.518+0000 7ff5b782a640 1 -- start start 2024-04-17T06:10:28.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b782a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b00d19e0 0x7ff5b00c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b782a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5b00d43d0 0x7ff5b01f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b782a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5b015ec00 0x7ff5b0161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b782a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b00d2ed0 con 0x7ff5b00d19e0 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b782a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b00d3bd0 con 0x7ff5b015ec00 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b782a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b00d3050 con 0x7ff5b00d43d0 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b00d19e0 0x7ff5b00c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b00d19e0 0x7ff5b00c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54714/0 (socket says 172.21.15.135:54714) 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 -- 172.21.15.135:0/1874945722 learned_addr learned my addr 172.21.15.135:0/1874945722 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b4dc5640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5b00d43d0 0x7ff5b01f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b5dc7640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5b015ec00 0x7ff5b0161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 -- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5b00d43d0 msgr2=0x7ff5b01f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5b00d43d0 0x7ff5b01f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 -- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5b015ec00 msgr2=0x7ff5b0161290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5b015ec00 0x7ff5b0161290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 -- 172.21.15.135:0/1874945722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b00c9cd0 con 0x7ff5b00d19e0 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b4dc5640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5b00d43d0 0x7ff5b01f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b5dc7640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5b015ec00 0x7ff5b0161290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b55c6640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b00d19e0 0x7ff5b00c96f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009960 tx=0x7ff5ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5a5ffb640 1 -- 172.21.15.135:0/1874945722 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac047070 con 0x7ff5b00d19e0 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5a5ffb640 1 -- 172.21.15.135:0/1874945722 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac037c40 con 0x7ff5b00d19e0 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b00c9fb0 con 0x7ff5b00d19e0 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5a5ffb640 1 -- 172.21.15.135:0/1874945722 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac004030 con 0x7ff5b00d19e0 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.519+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b00ca570 con 0x7ff5b00d19e0 2024-04-17T06:10:28.521 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.521+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b0074940 con 0x7ff5b00d19e0 2024-04-17T06:10:28.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.521+0000 7ff5a5ffb640 1 -- 172.21.15.135:0/1874945722 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5ac04a070 con 0x7ff5b00d19e0 2024-04-17T06:10:28.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.521+0000 7ff5a5ffb640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff59406c8f0 0x7ff59406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.522 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.522+0000 7ff5b4dc5640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff59406c8f0 0x7ff59406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.522+0000 7ff5a5ffb640 1 -- 172.21.15.135:0/1874945722 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6101+0+0 (secure 0 0 0) 0x7ff5ac0adda0 con 0x7ff5b00d19e0 2024-04-17T06:10:28.523 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.522+0000 7ff5b4dc5640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff59406c8f0 0x7ff59406edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff5b01542f0 tx=0x7ff5a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.524 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.526+0000 7ff5a5ffb640 1 -- 172.21.15.135:0/1874945722 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5ac033030 con 0x7ff5b00d19e0 2024-04-17T06:10:28.736 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.738+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff5b00b5d80 con 0x7ff5b00d19e0 2024-04-17T06:10:28.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.784+0000 7ff5a5ffb640 1 -- 172.21.15.135:0/1874945722 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7ff5ac07c0d0 con 0x7ff5b00d19e0 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff59406c8f0 msgr2=0x7ff59406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff59406c8f0 0x7ff59406edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff5b01542f0 tx=0x7ff5a0009040 comp rx=0 tx=0).stop 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b00d19e0 msgr2=0x7ff5b00c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b00d19e0 0x7ff5b00c96f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009960 tx=0x7ff5ac004730 comp rx=0 tx=0).stop 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 shutdown_connections 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff5b015ec00 0x7ff5b0161290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5b00d43d0 0x7ff5b01f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7ff59406c8f0 0x7ff59406edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.785 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 --2- 172.21.15.135:0/1874945722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5b00d19e0 0x7ff5b00c96f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 >> 172.21.15.135:0/1874945722 conn(0x7ff5b006cc90 msgr2=0x7ff5b00b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:28.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 shutdown_connections 2024-04-17T06:10:28.786 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.787+0000 7ff5b782a640 1 -- 172.21.15.135:0/1874945722 wait complete. 2024-04-17T06:10:28.796 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-17T06:10:28.796 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-17T06:10:28.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.903+0000 7f9394850640 1 Processor -- start 2024-04-17T06:10:28.901 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.903+0000 7f9394850640 1 -- start start 2024-04-17T06:10:28.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.904+0000 7f9394850640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 0x7f93900af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.904+0000 7f9394850640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f93900afc00 0x7f93900ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.904+0000 7f9394850640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93900b3a50 con 0x7f939015bb00 2024-04-17T06:10:28.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.904+0000 7f9394850640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f93900b5860 con 0x7f93900afc00 2024-04-17T06:10:28.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.904+0000 7f9394850640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f93900b0080 con 0x7f93900b3590 2024-04-17T06:10:28.905 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.904+0000 7f938e59c640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f939015bb00 0x7f93900b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41852/0 (socket says 172.21.15.135:41852) 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.904+0000 7f938e59c640 1 -- 172.21.15.135:0/2216883498 learned_addr learned my addr 172.21.15.135:0/2216883498 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2700102589 0 0) 0x7f93900b3a50 con 0x7f939015bb00 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938dd9b640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 0x7f93900af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938ed9d640 1 --2- 172.21.15.135:0/2216883498 >> v2:172.21.15.190:3300/0 conn(0x7f93900afc00 0x7f93900ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f937c003520 con 0x7f939015bb00 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3964034207 0 0) 0x7f937c003520 con 0x7f939015bb00 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 msgr2=0x7f93900af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 0x7f93900af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 >> v2:172.21.15.190:3300/0 conn(0x7f93900afc00 msgr2=0x7f93900ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 --2- 172.21.15.135:0/2216883498 >> v2:172.21.15.190:3300/0 conn(0x7f93900afc00 0x7f93900ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93900b3d30 con 0x7f939015bb00 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f93880019f0 con 0x7f939015bb00 2024-04-17T06:10:28.906 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 >> v1:172.21.15.135:6789/0 conn(0x7f939015bb00 legacy=0x7f93900b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f937c0039b0 0x7f937c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c004800 0x7f937c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 0x7f937c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93900b5860 con 0x7f937c0039b0 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93900b0080 con 0x7f937c004800 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f937c00b9c0 con 0x7f93900b3590 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938e59c640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f937c0039b0 0x7f937c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938dd9b640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c004800 0x7f937c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938ed9d640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 0x7f937c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938e59c640 1 -- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 msgr2=0x7f937c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938e59c640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 0x7f937c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938e59c640 1 -- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c004800 msgr2=0x7f937c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938e59c640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c004800 0x7f937c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938e59c640 1 -- 172.21.15.135:0/2216883498 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c003520 con 0x7f937c0039b0 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938dd9b640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c004800 0x7f937c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.905+0000 7f938ed9d640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93900b3590 0x7f937c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.906+0000 7f938e59c640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f937c0039b0 0x7f937c003dd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f938800a870 tx=0x7f937c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcd084d14847d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.906+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800a4c0 con 0x7f937c0039b0 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.906+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9388034630 con 0x7f937c0039b0 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.906+0000 7f938d59a640 1 -- 172.21.15.135:0/2216883498 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388034a20 con 0x7f937c0039b0 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.907+0000 7f9394850640 1 -- 172.21.15.135:0/2216883498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f937c0039b0 msgr2=0x7f937c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.907+0000 7f9394850640 1 --2- 172.21.15.135:0/2216883498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f937c0039b0 0x7f937c003dd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f938800a870 tx=0x7f937c005830 comp rx=0 tx=0).stop 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.907+0000 7f938ed9d640 1 -- 172.21.15.135:0/2216883498 reap_dead start 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.907+0000 7f9394850640 1 -- 172.21.15.135:0/2216883498 shutdown_connections 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.907+0000 7f9394850640 1 -- 172.21.15.135:0/2216883498 >> 172.21.15.135:0/2216883498 conn(0x7f93900d0520 msgr2=0x7f93900d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.907+0000 7f9394850640 1 -- 172.21.15.135:0/2216883498 shutdown_connections 2024-04-17T06:10:28.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.907+0000 7f9394850640 1 -- 172.21.15.135:0/2216883498 wait complete. 2024-04-17T06:10:28.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.908+0000 7f9394850640 1 Processor -- start 2024-04-17T06:10:28.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.908+0000 7f9394850640 1 -- start start 2024-04-17T06:10:28.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f9394850640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0039b0 0x7f9390145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f9394850640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f939015d190 0x7f93901558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f9394850640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f939015d610 0x7f9390155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f9394850640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93900b42f0 con 0x7f939015d610 2024-04-17T06:10:28.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f9394850640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f93900b4660 con 0x7f937c0039b0 2024-04-17T06:10:28.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f9394850640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f93900af6c0 con 0x7f939015d190 2024-04-17T06:10:28.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f939015d610 0x7f9390155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f939015d610 0x7f9390155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54724/0 (socket says 172.21.15.135:54724) 2024-04-17T06:10:28.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938ed9d640 1 -- 172.21.15.135:0/2617973868 learned_addr learned my addr 172.21.15.135:0/2617973868 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:28.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938e59c640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0039b0 0x7f9390145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938dd9b640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f939015d190 0x7f93901558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938ed9d640 1 -- 172.21.15.135:0/2617973868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f939015d190 msgr2=0x7f93901558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938ed9d640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f939015d190 0x7f93901558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938ed9d640 1 -- 172.21.15.135:0/2617973868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0039b0 msgr2=0x7f9390145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938ed9d640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0039b0 0x7f9390145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.909+0000 7f938ed9d640 1 -- 172.21.15.135:0/2617973868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9390155110 con 0x7f939015d610 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.910+0000 7f938ed9d640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f939015d610 0x7f9390155e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f939006de40 tx=0x7f9380009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.910+0000 7f9386ffd640 1 -- 172.21.15.135:0/2617973868 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380019020 con 0x7f939015d610 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.910+0000 7f9386ffd640 1 -- 172.21.15.135:0/2617973868 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9380004440 con 0x7f939015d610 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.910+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93901e3f10 con 0x7f939015d610 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.910+0000 7f9386ffd640 1 -- 172.21.15.135:0/2617973868 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380003850 con 0x7f939015d610 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.910+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93901e44d0 con 0x7f939015d610 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.911+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93900716c0 con 0x7f939015d610 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.911+0000 7f9386ffd640 1 -- 172.21.15.135:0/2617973868 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f938001c020 con 0x7f939015d610 2024-04-17T06:10:28.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.912+0000 7f9386ffd640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f937006c8f0 0x7f937006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:28.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.912+0000 7f9386ffd640 1 -- 172.21.15.135:0/2617973868 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f938008bf60 con 0x7f939015d610 2024-04-17T06:10:28.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.912+0000 7f938e59c640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f937006c8f0 0x7f937006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:28.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.912+0000 7f938e59c640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f937006c8f0 0x7f937006edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9388034000 tx=0x7f9388009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:28.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:28.916+0000 7f9386ffd640 1 -- 172.21.15.135:0/2617973868 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93900716c0 con 0x7f939015d610 2024-04-17T06:10:29.127 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.129+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9390074940 con 0x7f939015d610 2024-04-17T06:10:29.787 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.789+0000 7f9386ffd640 1 -- 172.21.15.135:0/2617973868 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f9380023030 con 0x7f939015d610 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f937006c8f0 msgr2=0x7f937006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f937006c8f0 0x7f937006edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9388034000 tx=0x7f9388009030 comp rx=0 tx=0).stop 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f939015d610 msgr2=0x7f9390155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f939015d610 0x7f9390155e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f939006de40 tx=0x7f9380009f90 comp rx=0 tx=0).stop 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 shutdown_connections 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f937006c8f0 0x7f937006edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f939015d610 0x7f9390155e20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f939015d190 0x7f93901558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 --2- 172.21.15.135:0/2617973868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f937c0039b0 0x7f9390145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.791+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 >> 172.21.15.135:0/2617973868 conn(0x7f93900d0520 msgr2=0x7f93900ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:29.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.792+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 shutdown_connections 2024-04-17T06:10:29.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.792+0000 7f9394850640 1 -- 172.21.15.135:0/2617973868 wait complete. 2024-04-17T06:10:29.803 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-17T06:10:29.803 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:10:29.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.910+0000 7fcb79d89640 1 Processor -- start 2024-04-17T06:10:29.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.910+0000 7fcb79d89640 1 -- start start 2024-04-17T06:10:29.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.910+0000 7fcb79d89640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcb7415bb00 0x7fcb740b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.910+0000 7fcb79d89640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcb740b3590 0x7fcb740af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.910+0000 7fcb79d89640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb740b3a50 con 0x7fcb740afc00 2024-04-17T06:10:29.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.910+0000 7fcb79d89640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fcb740b4660 con 0x7fcb7415bb00 2024-04-17T06:10:29.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.910+0000 7fcb79d89640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fcb740b3d30 con 0x7fcb740b3590 2024-04-17T06:10:29.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb737fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcb7415bb00 0x7fcb740b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.909 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb73fff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fcb740afc00 0x7fcb74071ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41854/0 (socket says 172.21.15.135:41854) 2024-04-17T06:10:29.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb73fff640 1 -- 172.21.15.135:0/247988810 learned_addr learned my addr 172.21.15.135:0/247988810 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:29.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb737fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fcb7415bb00 0x7fcb740b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54732/0 (socket says 172.21.15.135:54732) 2024-04-17T06:10:29.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb72ffd640 1 --2- 172.21.15.135:0/247988810 >> v2:172.21.15.190:3300/0 conn(0x7fcb740b3590 0x7fcb740af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb737fe640 1 -- 172.21.15.135:0/247988810 >> v2:172.21.15.190:3300/0 conn(0x7fcb740b3590 msgr2=0x7fcb740af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:29.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb737fe640 1 --2- 172.21.15.135:0/247988810 >> v2:172.21.15.190:3300/0 conn(0x7fcb740b3590 0x7fcb740af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb737fe640 1 -- 172.21.15.135:0/247988810 >> v1:172.21.15.135:6789/0 conn(0x7fcb740afc00 legacy=0x7fcb74071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.911+0000 7fcb737fe640 1 -- 172.21.15.135:0/247988810 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb740b4010 con 0x7fcb7415bb00 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb737fe640 1 --2- 172.21.15.135:0/247988810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb740b5150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb7406dd80 tx=0x7fcb6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e7d0b848d388aa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb727fc640 1 -- 172.21.15.135:0/247988810 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64035020 con 0x7fcb7415bb00 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb727fc640 1 -- 172.21.15.135:0/247988810 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb64031830 con 0x7fcb7415bb00 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb727fc640 1 -- 172.21.15.135:0/247988810 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64031c00 con 0x7fcb7415bb00 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb79d89640 1 -- 172.21.15.135:0/247988810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 msgr2=0x7fcb740b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb79d89640 1 --2- 172.21.15.135:0/247988810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb740b5150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcb7406dd80 tx=0x7fcb6402b8d0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb79d89640 1 -- 172.21.15.135:0/247988810 shutdown_connections 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb79d89640 1 --2- 172.21.15.135:0/247988810 >> v2:172.21.15.190:3300/0 conn(0x7fcb740b3590 0x7fcb740af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb79d89640 1 --2- 172.21.15.135:0/247988810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb740b5150 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb79d89640 1 -- 172.21.15.135:0/247988810 >> 172.21.15.135:0/247988810 conn(0x7fcb740d0520 msgr2=0x7fcb740d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:29.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb79d89640 1 -- 172.21.15.135:0/247988810 shutdown_connections 2024-04-17T06:10:29.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.912+0000 7fcb79d89640 1 -- 172.21.15.135:0/247988810 wait complete. 2024-04-17T06:10:29.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.914+0000 7fcb79d89640 1 Processor -- start 2024-04-17T06:10:29.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.914+0000 7fcb79d89640 1 -- start start 2024-04-17T06:10:29.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.914+0000 7fcb79d89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb740afc00 0x7fcb74145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.914+0000 7fcb79d89640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb740b3590 0x7fcb741f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.914+0000 7fcb79d89640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb741f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.914+0000 7fcb79d89640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb740b5810 con 0x7fcb7415bb00 2024-04-17T06:10:29.916 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.914+0000 7fcb79d89640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fcb740b5690 con 0x7fcb740b3590 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.914+0000 7fcb79d89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fcb740cf730 con 0x7fcb740afc00 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb741f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb741f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54738/0 (socket says 172.21.15.135:54738) 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 -- 172.21.15.135:0/219769983 learned_addr learned my addr 172.21.15.135:0/219769983 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb72ffd640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb740b3590 0x7fcb741f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb737fe640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb740afc00 0x7fcb74145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 -- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb740afc00 msgr2=0x7fcb74145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb740afc00 0x7fcb74145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 -- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb740b3590 msgr2=0x7fcb741f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb740b3590 0x7fcb741f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 -- 172.21.15.135:0/219769983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb741f5450 con 0x7fcb7415bb00 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb72ffd640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb740b3590 0x7fcb741f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:29.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb73fff640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb741f4e60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00cbc0 tx=0x7fcb6c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb5bfff640 1 -- 172.21.15.135:0/219769983 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c017070 con 0x7fcb7415bb00 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb5bfff640 1 -- 172.21.15.135:0/219769983 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb6c00ad40 con 0x7fcb7415bb00 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb741f5730 con 0x7fcb7415bb00 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb5bfff640 1 -- 172.21.15.135:0/219769983 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c012600 con 0x7fcb7415bb00 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.915+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7414f700 con 0x7fcb7415bb00 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.916+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb74074940 con 0x7fcb7415bb00 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.917+0000 7fcb5bfff640 1 -- 172.21.15.135:0/219769983 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb6c01a070 con 0x7fcb7415bb00 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.917+0000 7fcb5bfff640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcb6006c8f0 0x7fcb6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:29.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.917+0000 7fcb5bfff640 1 -- 172.21.15.135:0/219769983 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fcb6c08b980 con 0x7fcb7415bb00 2024-04-17T06:10:29.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.917+0000 7fcb737fe640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcb6006c8f0 0x7fcb6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:29.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.918+0000 7fcb737fe640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcb6006c8f0 0x7fcb6006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb64036000 tx=0x7fcb64008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:29.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:29.922+0000 7fcb5bfff640 1 -- 172.21.15.135:0/219769983 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb6c0595d0 con 0x7fcb7415bb00 2024-04-17T06:10:30.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.135+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb740af290 con 0x7fcb7415bb00 2024-04-17T06:10:30.133 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.135+0000 7fcb5bfff640 1 -- 172.21.15.135:0/219769983 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7fcb6c05d8d0 con 0x7fcb7415bb00 2024-04-17T06:10:30.133 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:30.134 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1269381293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1269381293},{"type":"v1","addr":"172.21.15.190:6834","nonce":1269381293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/354919697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":354919697},{"type":"v1","addr":"172.21.15.190:6835","nonce":354919697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1997531955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1997531955},{"type":"v1","addr":"172.21.15.135:6836","nonce":1997531955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:10:27.119795+0000","modified":"2024-04-17T06:10:29.130787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/3309431937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3309431937},{"type":"v1","addr":"172.21.15.135:6837","nonce":3309431937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5379,"qdb_cluster":[5379]},"id":2}]} 2024-04-17T06:10:30.134 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 16 2024-04-17T06:10:30.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcb6006c8f0 msgr2=0x7fcb6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcb6006c8f0 0x7fcb6006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb64036000 tx=0x7fcb64008030 comp rx=0 tx=0).stop 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 msgr2=0x7fcb741f4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb741f4e60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c00cbc0 tx=0x7fcb6c00a430 comp rx=0 tx=0).stop 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 shutdown_connections 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fcb6006c8f0 0x7fcb6006edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb7415bb00 0x7fcb741f4e60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb740b3590 0x7fcb741f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 --2- 172.21.15.135:0/219769983 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb740afc00 0x7fcb74145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 >> 172.21.15.135:0/219769983 conn(0x7fcb740d0520 msgr2=0x7fcb74071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 shutdown_connections 2024-04-17T06:10:30.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.137+0000 7fcb79d89640 1 -- 172.21.15.135:0/219769983 wait complete. 2024-04-17T06:10:30.146 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:10:30.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.255+0000 7fa48c9e9640 1 Processor -- start 2024-04-17T06:10:30.253 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.255+0000 7fa48c9e9640 1 -- start start 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa48c9e9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48815cac0 0x7fa48806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa48c9e9640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa4880d03d0 0x7fa4880add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa48c9e9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa48815cea0 con 0x7fa48806d020 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa48c9e9640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fa4880d0940 con 0x7fa4880d03d0 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa48c9e9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa48806c540 con 0x7fa48815cac0 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa486d9d640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fa4880d03d0 0x7fa4880add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa47ffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fa48806d020 0x7fa488070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41870/0 (socket says 172.21.15.135:41870) 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa47ffff640 1 -- 172.21.15.135:0/445198535 learned_addr learned my addr 172.21.15.135:0/445198535 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa486d9d640 1 -- 172.21.15.135:0/445198535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48815cac0 msgr2=0x7fa48806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa486d9d640 1 --2- 172.21.15.135:0/445198535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48815cac0 0x7fa48806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.255 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa486d9d640 1 -- 172.21.15.135:0/445198535 >> v1:172.21.15.135:6789/0 conn(0x7fa48806d020 legacy=0x7fa488070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa486d9d640 1 -- 172.21.15.135:0/445198535 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48806c820 con 0x7fa4880d03d0 2024-04-17T06:10:30.256 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.256+0000 7fa485d9b640 1 -- 172.21.15.135:0/445198535 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3335806155 0 0) 0x7fa48815cea0 con 0x7fa48806d020 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.257+0000 7fa486d9d640 1 --2- 172.21.15.135:0/445198535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4880add30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa480009d90 tx=0x7fa48002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354af291c1cb8e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.257+0000 7fa485d9b640 1 -- 172.21.15.135:0/445198535 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480039070 con 0x7fa4880d03d0 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.257+0000 7fa485d9b640 1 -- 172.21.15.135:0/445198535 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48002cc50 con 0x7fa4880d03d0 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.257+0000 7fa485d9b640 1 -- 172.21.15.135:0/445198535 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480034440 con 0x7fa4880d03d0 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.258+0000 7fa48c9e9640 1 -- 172.21.15.135:0/445198535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 msgr2=0x7fa4880add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.258+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/445198535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4880add30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa480009d90 tx=0x7fa48002bea0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.258+0000 7fa48c9e9640 1 -- 172.21.15.135:0/445198535 shutdown_connections 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.258+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/445198535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4880add30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.257 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.258+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/445198535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48815cac0 0x7fa48806e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.258+0000 7fa48c9e9640 1 -- 172.21.15.135:0/445198535 >> 172.21.15.135:0/445198535 conn(0x7fa4880cd7e0 msgr2=0x7fa4880cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:30.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.258+0000 7fa48c9e9640 1 -- 172.21.15.135:0/445198535 shutdown_connections 2024-04-17T06:10:30.258 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.258+0000 7fa48c9e9640 1 -- 172.21.15.135:0/445198535 wait complete. 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.259+0000 7fa48c9e9640 1 Processor -- start 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.259+0000 7fa48c9e9640 1 -- start start 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa48c9e9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48806d020 0x7fa488157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa48c9e9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4881f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa48c9e9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa48815cac0 0x7fa4881f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa48c9e9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa48806ef10 con 0x7fa4880d03d0 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa48c9e9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fa48806ed90 con 0x7fa48815cac0 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa48c9e9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fa488070a80 con 0x7fa48806d020 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa47ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4881f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa47ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4881f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54754/0 (socket says 172.21.15.135:54754) 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa47ffff640 1 -- 172.21.15.135:0/3557441076 learned_addr learned my addr 172.21.15.135:0/3557441076 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:30.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa47ffff640 1 -- 172.21.15.135:0/3557441076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48806d020 msgr2=0x7fa488157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa486d9d640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa48815cac0 0x7fa4881f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa48659c640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48806d020 0x7fa488157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa47ffff640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48806d020 0x7fa488157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa47ffff640 1 -- 172.21.15.135:0/3557441076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa48815cac0 msgr2=0x7fa4881f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa47ffff640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa48815cac0 0x7fa4881f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.260+0000 7fa47ffff640 1 -- 172.21.15.135:0/3557441076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48809b4a0 con 0x7fa4880d03d0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.261+0000 7fa47ffff640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4881f5a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa47000c4d0 tx=0x7fa47000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.261+0000 7fa47effd640 1 -- 172.21.15.135:0/3557441076 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470018070 con 0x7fa4880d03d0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.261+0000 7fa47effd640 1 -- 172.21.15.135:0/3557441076 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470007500 con 0x7fa4880d03d0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.261+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48809b780 con 0x7fa4880d03d0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.261+0000 7fa47effd640 1 -- 172.21.15.135:0/3557441076 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470004a20 con 0x7fa4880d03d0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.261+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48809bd40 con 0x7fa4880d03d0 2024-04-17T06:10:30.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.262+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa488074940 con 0x7fa4880d03d0 2024-04-17T06:10:30.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.262+0000 7fa47effd640 1 -- 172.21.15.135:0/3557441076 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa47001b070 con 0x7fa4880d03d0 2024-04-17T06:10:30.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.262+0000 7fa47effd640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.263+0000 7fa47effd640 1 -- 172.21.15.135:0/3557441076 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fa47008afe0 con 0x7fa4880d03d0 2024-04-17T06:10:30.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.263+0000 7fa48659c640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.263 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.263+0000 7fa48659c640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa46806c8f0 0x7fa46806edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa47800a770 tx=0x7fa478008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.267+0000 7fa47effd640 1 -- 172.21.15.135:0/3557441076 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa470059280 con 0x7fa4880d03d0 2024-04-17T06:10:30.440 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.442+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4880af520 con 0x7fa4880d03d0 2024-04-17T06:10:30.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.444+0000 7fa47effd640 1 -- 172.21.15.135:0/3557441076 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16990 (secure 0 0 0) 0x7fa470058b80 con 0x7fa4880d03d0 2024-04-17T06:10:30.442 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:30.442 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":38,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:10:28.781026+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:10:25.616330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-17T06:10:26.077318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:10:30.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa46806c8f0 msgr2=0x7fa46806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa46806c8f0 0x7fa46806edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa47800a770 tx=0x7fa478008040 comp rx=0 tx=0).stop 2024-04-17T06:10:30.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 msgr2=0x7fa4881f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4881f5a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa47000c4d0 tx=0x7fa47000c990 comp rx=0 tx=0).stop 2024-04-17T06:10:30.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 shutdown_connections 2024-04-17T06:10:30.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa48815cac0 0x7fa4881f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fa46806c8f0 0x7fa46806edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4880d03d0 0x7fa4881f5a20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 --2- 172.21.15.135:0/3557441076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa48806d020 0x7fa488157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 >> 172.21.15.135:0/3557441076 conn(0x7fa4880cd7e0 msgr2=0x7fa4880ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:30.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 shutdown_connections 2024-04-17T06:10:30.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.446+0000 7fa48c9e9640 1 -- 172.21.15.135:0/3557441076 wait complete. 2024-04-17T06:10:30.455 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-17T06:10:30.455 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T06:10:30.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.561+0000 7f71a0cd9640 1 Processor -- start 2024-04-17T06:10:30.559 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.561+0000 7f71a0cd9640 1 -- start start 2024-04-17T06:10:30.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f71a0cd9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f719c0afdd0 0x7f719c1664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.562 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f71a0cd9640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719c0b3a50 con 0x7f719c0b3590 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f71a0cd9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f719c0b3d30 con 0x7f719c15bb00 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f71a0cd9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f719c0b5860 con 0x7f719c0afdd0 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f7199d9b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f719c0b3590 0x7f719c0af6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41874/0 (socket says 172.21.15.135:41874) 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f7199d9b640 1 -- 172.21.15.135:0/2397656941 learned_addr learned my addr 172.21.15.135:0/2397656941 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719ad9d640 1 --2- 172.21.15.135:0/2397656941 >> v2:172.21.15.190:3300/0 conn(0x7f719c0afdd0 0x7f719c1664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644130953 0 0) 0x7f719c0b3a50 con 0x7f719c0b3590 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7188003520 con 0x7f719c0b3590 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160223347 0 0) 0x7f719c0b3d30 con 0x7f719c15bb00 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f719c0b3a50 con 0x7f719c15bb00 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3804143922 0 0) 0x7f7188003520 con 0x7f719c0b3590 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 >> v2:172.21.15.190:3300/0 conn(0x7f719c0afdd0 msgr2=0x7f719c1664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 --2- 172.21.15.135:0/2397656941 >> v2:172.21.15.190:3300/0 conn(0x7f719c0afdd0 0x7f719c1664e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 >> v1:172.21.15.190:6789/0 conn(0x7f719c15bb00 legacy=0x7f719c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.563 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c0b4010 con 0x7f719c0b3590 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f71840015f0 con 0x7f719c0b3590 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 >> v1:172.21.15.135:6789/0 conn(0x7f719c0b3590 legacy=0x7f719c0af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.562+0000 7f719959a640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7188003b90 0x7f7188003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719959a640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7188004800 0x7f7188004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719959a640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71880051c0 0x7f718800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f719c0b5860 con 0x7f71880051c0 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f718800c090 con 0x7f7188003b90 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f718800c290 con 0x7f7188004800 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719ad9d640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71880051c0 0x7f718800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719a59c640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7188003b90 0x7f7188003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f7199d9b640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7188004800 0x7f7188004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719ad9d640 1 -- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7188004800 msgr2=0x7f7188004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719ad9d640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7188004800 0x7f7188004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.564 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719ad9d640 1 -- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7188003b90 msgr2=0x7f7188003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719ad9d640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7188003b90 0x7f7188003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f719ad9d640 1 -- 172.21.15.135:0/2397656941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7188003520 con 0x7f71880051c0 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.563+0000 7f7199d9b640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7188004800 0x7f7188004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f719ad9d640 1 -- 172.21.15.135:0/2397656941 reap_dead start 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f719ad9d640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71880051c0 0x7f718800b9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f718c009a90 tx=0x7f7188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14bf5d3dec3f275 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c039070 con 0x7f71880051c0 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718c02c6b0 con 0x7f71880051c0 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f719959a640 1 -- 172.21.15.135:0/2397656941 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c009690 con 0x7f71880051c0 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f71a0cd9640 1 -- 172.21.15.135:0/2397656941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71880051c0 msgr2=0x7f718800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f71a0cd9640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71880051c0 0x7f718800b9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f718c009a90 tx=0x7f7188005830 comp rx=0 tx=0).stop 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f71a0cd9640 1 -- 172.21.15.135:0/2397656941 shutdown_connections 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f71a0cd9640 1 --2- 172.21.15.135:0/2397656941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71880051c0 0x7f718800b9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f71a0cd9640 1 -- 172.21.15.135:0/2397656941 >> 172.21.15.135:0/2397656941 conn(0x7f719c0d0520 msgr2=0x7f719c0d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:30.565 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f71a0cd9640 1 -- 172.21.15.135:0/2397656941 shutdown_connections 2024-04-17T06:10:30.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.564+0000 7f71a0cd9640 1 -- 172.21.15.135:0/2397656941 wait complete. 2024-04-17T06:10:30.568 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.566+0000 7f71a0cd9640 1 Processor -- start 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.566+0000 7f71a0cd9640 1 -- start start 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.566+0000 7f71a0cd9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71880051c0 0x7f719c14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.566+0000 7f71a0cd9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c143510 0x7f719c143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.566+0000 7f71a0cd9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f719c13acc0 0x7f719c13b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.566+0000 7f71a0cd9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f719c0af6c0 con 0x7f719c13acc0 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.566+0000 7f71a0cd9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f719c0b4660 con 0x7f71880051c0 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.566+0000 7f71a0cd9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f719c0af840 con 0x7f719c143510 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f719c13acc0 0x7f719c13b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f719c13acc0 0x7f719c13b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54768/0 (socket says 172.21.15.135:54768) 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 -- 172.21.15.135:0/1898012282 learned_addr learned my addr 172.21.15.135:0/1898012282 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719a59c640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71880051c0 0x7f719c14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f7199d9b640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c143510 0x7f719c143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 -- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c143510 msgr2=0x7f719c143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c143510 0x7f719c143990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 -- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71880051c0 msgr2=0x7f719c14fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71880051c0 0x7f719c14fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 -- 172.21.15.135:0/1898012282 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c150410 con 0x7f719c13acc0 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719a59c640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71880051c0 0x7f719c14fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f719ad9d640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f719c13acc0 0x7f719c13b170 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f718c009a90 tx=0x7f718c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f7192ffd640 1 -- 172.21.15.135:0/1898012282 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c046070 con 0x7f719c13acc0 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f7192ffd640 1 -- 172.21.15.135:0/1898012282 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718c03e890 con 0x7f719c13acc0 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c1506d0 con 0x7f719c13acc0 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f7192ffd640 1 -- 172.21.15.135:0/1898012282 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c039040 con 0x7f719c13acc0 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.567+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c150c90 con 0x7f719c13acc0 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.568+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c0afb70 con 0x7f719c13acc0 2024-04-17T06:10:30.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.569+0000 7f7192ffd640 1 -- 172.21.15.135:0/1898012282 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f718c049070 con 0x7f719c13acc0 2024-04-17T06:10:30.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.569+0000 7f7192ffd640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f717c06c920 0x7f717c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.569+0000 7f7192ffd640 1 -- 172.21.15.135:0/1898012282 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f718c0aca00 con 0x7f719c13acc0 2024-04-17T06:10:30.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.569+0000 7f719a59c640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f717c06c920 0x7f717c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.569+0000 7f719a59c640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f717c06c920 0x7f717c06ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f719c144100 tx=0x7f7194009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.571 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.574+0000 7f7192ffd640 1 -- 172.21.15.135:0/1898012282 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f719c0afb70 con 0x7f719c13acc0 2024-04-17T06:10:30.742 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.744+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 --> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f719c074940 con 0x7f717c06c920 2024-04-17T06:10:30.748 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.750+0000 7f7192ffd640 1 -- 172.21.15.135:0/1898012282 <== mgr.4113 v2:172.21.15.135:6832/3190814219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251887 (secure 0 0 0) 0x7f719c074940 con 0x7f717c06c920 2024-04-17T06:10:30.751 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:30.753 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-17T06:10:30.754 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f717c06c920 msgr2=0x7f717c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f717c06c920 0x7f717c06ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f719c144100 tx=0x7f7194009040 comp rx=0 tx=0).stop 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f719c13acc0 msgr2=0x7f719c13b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f719c13acc0 0x7f719c13b170 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f718c009a90 tx=0x7f718c004730 comp rx=0 tx=0).stop 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 shutdown_connections 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f717c06c920 0x7f717c06ede0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f719c13acc0 0x7f719c13b170 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c143510 0x7f719c143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 --2- 172.21.15.135:0/1898012282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71880051c0 0x7f719c14fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 >> 172.21.15.135:0/1898012282 conn(0x7f719c0d0520 msgr2=0x7f719c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:30.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 shutdown_connections 2024-04-17T06:10:30.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.753+0000 7f71a0cd9640 1 -- 172.21.15.135:0/1898012282 wait complete. 2024-04-17T06:10:30.763 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-17T06:10:29.951283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220584,"kb_used_data":4408,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754136,"statfs":{"total":773094113280,"available":772868235264,"internally_reserved":0,"allocated":4513792,"data_stored":2597261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.192974"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797951+0000","last_change":"2024-04-17T06:10:27.793433+0000","last_active":"2024-04-17T06:10:27.797951+0000","last_peered":"2024-04-17T06:10:27.797951+0000","last_clean":"2024-04-17T06:10:27.797951+0000","last_became_active":"2024-04-17T06:10:27.792612+0000","last_became_peered":"2024-04-17T06:10:27.792612+0000","last_unstale":"2024-04-17T06:10:27.797951+0000","last_undegraded":"2024-04-17T06:10:27.797951+0000","last_fullsized":"2024-04-17T06:10:27.797951+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:48.774295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797817+0000","last_change":"2024-04-17T06:10:27.793271+0000","last_active":"2024-04-17T06:10:27.797817+0000","last_peered":"2024-04-17T06:10:27.797817+0000","last_clean":"2024-04-17T06:10:27.797817+0000","last_became_active":"2024-04-17T06:10:27.792418+0000","last_became_peered":"2024-04-17T06:10:27.792418+0000","last_unstale":"2024-04-17T06:10:27.797817+0000","last_undegraded":"2024-04-17T06:10:27.797817+0000","last_fullsized":"2024-04-17T06:10:27.797817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:45.511079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792412+0000","last_change":"2024-04-17T06:10:27.788173+0000","last_active":"2024-04-17T06:10:27.792412+0000","last_peered":"2024-04-17T06:10:27.792412+0000","last_clean":"2024-04-17T06:10:27.792412+0000","last_became_active":"2024-04-17T06:10:27.787899+0000","last_became_peered":"2024-04-17T06:10:27.787899+0000","last_unstale":"2024-04-17T06:10:27.792412+0000","last_undegraded":"2024-04-17T06:10:27.792412+0000","last_fullsized":"2024-04-17T06:10:27.792412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:23.566483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.789742+0000","last_change":"2024-04-17T06:10:27.786687+0000","last_active":"2024-04-17T06:10:27.789742+0000","last_peered":"2024-04-17T06:10:27.789742+0000","last_clean":"2024-04-17T06:10:27.789742+0000","last_became_active":"2024-04-17T06:10:27.786431+0000","last_became_peered":"2024-04-17T06:10:27.786431+0000","last_unstale":"2024-04-17T06:10:27.789742+0000","last_undegraded":"2024-04-17T06:10:27.789742+0000","last_fullsized":"2024-04-17T06:10:27.789742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:29.646387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792347+0000","last_change":"2024-04-17T06:10:27.788301+0000","last_active":"2024-04-17T06:10:27.792347+0000","last_peered":"2024-04-17T06:10:27.792347+0000","last_clean":"2024-04-17T06:10:27.792347+0000","last_became_active":"2024-04-17T06:10:27.788062+0000","last_became_peered":"2024-04-17T06:10:27.788062+0000","last_unstale":"2024-04-17T06:10:27.792347+0000","last_undegraded":"2024-04-17T06:10:27.792347+0000","last_fullsized":"2024-04-17T06:10:27.792347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:51.392110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798664+0000","last_change":"2024-04-17T06:10:27.794854+0000","last_active":"2024-04-17T06:10:27.798664+0000","last_peered":"2024-04-17T06:10:27.798664+0000","last_clean":"2024-04-17T06:10:27.798664+0000","last_became_active":"2024-04-17T06:10:27.794595+0000","last_became_peered":"2024-04-17T06:10:27.794595+0000","last_unstale":"2024-04-17T06:10:27.798664+0000","last_undegraded":"2024-04-17T06:10:27.798664+0000","last_fullsized":"2024-04-17T06:10:27.798664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:51.428907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.802315+0000","last_change":"2024-04-17T06:10:27.796909+0000","last_active":"2024-04-17T06:10:27.802315+0000","last_peered":"2024-04-17T06:10:27.802315+0000","last_clean":"2024-04-17T06:10:27.802315+0000","last_became_active":"2024-04-17T06:10:27.794908+0000","last_became_peered":"2024-04-17T06:10:27.794908+0000","last_unstale":"2024-04-17T06:10:27.802315+0000","last_undegraded":"2024-04-17T06:10:27.802315+0000","last_fullsized":"2024-04-17T06:10:27.802315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:48.399109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797556+0000","last_change":"2024-04-17T06:10:27.792851+0000","last_active":"2024-04-17T06:10:27.797556+0000","last_peered":"2024-04-17T06:10:27.797556+0000","last_clean":"2024-04-17T06:10:27.797556+0000","last_became_active":"2024-04-17T06:10:27.792557+0000","last_became_peered":"2024-04-17T06:10:27.792557+0000","last_unstale":"2024-04-17T06:10:27.797556+0000","last_undegraded":"2024-04-17T06:10:27.797556+0000","last_fullsized":"2024-04-17T06:10:27.797556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:22:41.057096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792393+0000","last_change":"2024-04-17T06:10:27.786708+0000","last_active":"2024-04-17T06:10:27.792393+0000","last_peered":"2024-04-17T06:10:27.792393+0000","last_clean":"2024-04-17T06:10:27.792393+0000","last_became_active":"2024-04-17T06:10:27.786475+0000","last_became_peered":"2024-04-17T06:10:27.786475+0000","last_unstale":"2024-04-17T06:10:27.792393+0000","last_undegraded":"2024-04-17T06:10:27.792393+0000","last_fullsized":"2024-04-17T06:10:27.792393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:27.405253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792783+0000","last_change":"2024-04-17T06:10:27.789058+0000","last_active":"2024-04-17T06:10:27.792783+0000","last_peered":"2024-04-17T06:10:27.792783+0000","last_clean":"2024-04-17T06:10:27.792783+0000","last_became_active":"2024-04-17T06:10:27.788774+0000","last_became_peered":"2024-04-17T06:10:27.788774+0000","last_unstale":"2024-04-17T06:10:27.792783+0000","last_undegraded":"2024-04-17T06:10:27.792783+0000","last_fullsized":"2024-04-17T06:10:27.792783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:45.511079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.790507+0000","last_change":"2024-04-17T06:10:27.787093+0000","last_active":"2024-04-17T06:10:27.790507+0000","last_peered":"2024-04-17T06:10:27.790507+0000","last_clean":"2024-04-17T06:10:27.790507+0000","last_became_active":"2024-04-17T06:10:27.786834+0000","last_became_peered":"2024-04-17T06:10:27.786834+0000","last_unstale":"2024-04-17T06:10:27.790507+0000","last_undegraded":"2024-04-17T06:10:27.790507+0000","last_fullsized":"2024-04-17T06:10:27.790507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:22:41.057096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792296+0000","last_change":"2024-04-17T06:10:27.786171+0000","last_active":"2024-04-17T06:10:27.792296+0000","last_peered":"2024-04-17T06:10:27.792296+0000","last_clean":"2024-04-17T06:10:27.792296+0000","last_became_active":"2024-04-17T06:10:27.785894+0000","last_became_peered":"2024-04-17T06:10:27.785894+0000","last_unstale":"2024-04-17T06:10:27.792296+0000","last_undegraded":"2024-04-17T06:10:27.792296+0000","last_fullsized":"2024-04-17T06:10:27.792296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.793798+0000","last_change":"2024-04-17T06:10:27.789108+0000","last_active":"2024-04-17T06:10:27.793798+0000","last_peered":"2024-04-17T06:10:27.793798+0000","last_clean":"2024-04-17T06:10:27.793798+0000","last_became_active":"2024-04-17T06:10:27.788829+0000","last_became_peered":"2024-04-17T06:10:27.788829+0000","last_unstale":"2024-04-17T06:10:27.793798+0000","last_undegraded":"2024-04-17T06:10:27.793798+0000","last_fullsized":"2024-04-17T06:10:27.793798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:06.115386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.795958+0000","last_change":"2024-04-17T06:10:27.786698+0000","last_active":"2024-04-17T06:10:27.795958+0000","last_peered":"2024-04-17T06:10:27.795958+0000","last_clean":"2024-04-17T06:10:27.795958+0000","last_became_active":"2024-04-17T06:10:27.786273+0000","last_became_peered":"2024-04-17T06:10:27.786273+0000","last_unstale":"2024-04-17T06:10:27.795958+0000","last_undegraded":"2024-04-17T06:10:27.795958+0000","last_fullsized":"2024-04-17T06:10:27.795958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.532545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.784571+0000","last_change":"2024-04-17T06:10:27.778601+0000","last_active":"2024-04-17T06:10:27.784571+0000","last_peered":"2024-04-17T06:10:27.784571+0000","last_clean":"2024-04-17T06:10:27.784571+0000","last_became_active":"2024-04-17T06:10:27.778335+0000","last_became_peered":"2024-04-17T06:10:27.778335+0000","last_unstale":"2024-04-17T06:10:27.784571+0000","last_undegraded":"2024-04-17T06:10:27.784571+0000","last_fullsized":"2024-04-17T06:10:27.784571+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:13.377063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796278+0000","last_change":"2024-04-17T06:10:27.788159+0000","last_active":"2024-04-17T06:10:27.796278+0000","last_peered":"2024-04-17T06:10:27.796278+0000","last_clean":"2024-04-17T06:10:27.796278+0000","last_became_active":"2024-04-17T06:10:27.787877+0000","last_became_peered":"2024-04-17T06:10:27.787877+0000","last_unstale":"2024-04-17T06:10:27.796278+0000","last_undegraded":"2024-04-17T06:10:27.796278+0000","last_fullsized":"2024-04-17T06:10:27.796278+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:13.072810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797403+0000","last_change":"2024-04-17T06:10:27.788810+0000","last_active":"2024-04-17T06:10:27.797403+0000","last_peered":"2024-04-17T06:10:27.797403+0000","last_clean":"2024-04-17T06:10:27.797403+0000","last_became_active":"2024-04-17T06:10:27.788481+0000","last_became_peered":"2024-04-17T06:10:27.788481+0000","last_unstale":"2024-04-17T06:10:27.797403+0000","last_undegraded":"2024-04-17T06:10:27.797403+0000","last_fullsized":"2024-04-17T06:10:27.797403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:51.428907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.793141+0000","last_change":"2024-04-17T06:10:27.789579+0000","last_active":"2024-04-17T06:10:27.793141+0000","last_peered":"2024-04-17T06:10:27.793141+0000","last_clean":"2024-04-17T06:10:27.793141+0000","last_became_active":"2024-04-17T06:10:27.789109+0000","last_became_peered":"2024-04-17T06:10:27.789109+0000","last_unstale":"2024-04-17T06:10:27.793141+0000","last_undegraded":"2024-04-17T06:10:27.793141+0000","last_fullsized":"2024-04-17T06:10:27.793141+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.532545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792107+0000","last_change":"2024-04-17T06:10:27.787264+0000","last_active":"2024-04-17T06:10:27.792107+0000","last_peered":"2024-04-17T06:10:27.792107+0000","last_clean":"2024-04-17T06:10:27.792107+0000","last_became_active":"2024-04-17T06:10:27.787008+0000","last_became_peered":"2024-04-17T06:10:27.787008+0000","last_unstale":"2024-04-17T06:10:27.792107+0000","last_undegraded":"2024-04-17T06:10:27.792107+0000","last_fullsized":"2024-04-17T06:10:27.792107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:32:14.211303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.789487+0000","last_change":"2024-04-17T06:10:27.787860+0000","last_active":"2024-04-17T06:10:27.789487+0000","last_peered":"2024-04-17T06:10:27.789487+0000","last_clean":"2024-04-17T06:10:27.789487+0000","last_became_active":"2024-04-17T06:10:27.787558+0000","last_became_peered":"2024-04-17T06:10:27.787558+0000","last_unstale":"2024-04-17T06:10:27.789487+0000","last_undegraded":"2024-04-17T06:10:27.789487+0000","last_fullsized":"2024-04-17T06:10:27.789487+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:48.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.790292+0000","last_change":"2024-04-17T06:10:27.784417+0000","last_active":"2024-04-17T06:10:27.790292+0000","last_peered":"2024-04-17T06:10:27.790292+0000","last_clean":"2024-04-17T06:10:27.790292+0000","last_became_active":"2024-04-17T06:10:27.784164+0000","last_became_peered":"2024-04-17T06:10:27.784164+0000","last_unstale":"2024-04-17T06:10:27.790292+0000","last_undegraded":"2024-04-17T06:10:27.790292+0000","last_fullsized":"2024-04-17T06:10:27.790292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:58.769842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.786605+0000","last_change":"2024-04-17T06:10:27.783246+0000","last_active":"2024-04-17T06:10:27.786605+0000","last_peered":"2024-04-17T06:10:27.786605+0000","last_clean":"2024-04-17T06:10:27.786605+0000","last_became_active":"2024-04-17T06:10:27.782711+0000","last_became_peered":"2024-04-17T06:10:27.782711+0000","last_unstale":"2024-04-17T06:10:27.786605+0000","last_undegraded":"2024-04-17T06:10:27.786605+0000","last_fullsized":"2024-04-17T06:10:27.786605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:30.540315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798059+0000","last_change":"2024-04-17T06:10:27.792547+0000","last_active":"2024-04-17T06:10:27.798059+0000","last_peered":"2024-04-17T06:10:27.798059+0000","last_clean":"2024-04-17T06:10:27.798059+0000","last_became_active":"2024-04-17T06:10:27.792302+0000","last_became_peered":"2024-04-17T06:10:27.792302+0000","last_unstale":"2024-04-17T06:10:27.798059+0000","last_undegraded":"2024-04-17T06:10:27.798059+0000","last_fullsized":"2024-04-17T06:10:27.798059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:23.566483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796106+0000","last_change":"2024-04-17T06:10:27.790879+0000","last_active":"2024-04-17T06:10:27.796106+0000","last_peered":"2024-04-17T06:10:27.796106+0000","last_clean":"2024-04-17T06:10:27.796106+0000","last_became_active":"2024-04-17T06:10:27.790347+0000","last_became_peered":"2024-04-17T06:10:27.790347+0000","last_unstale":"2024-04-17T06:10:27.796106+0000","last_undegraded":"2024-04-17T06:10:27.796106+0000","last_fullsized":"2024-04-17T06:10:27.796106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:34:11.210087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.784233+0000","last_change":"2024-04-17T06:10:27.777722+0000","last_active":"2024-04-17T06:10:27.784233+0000","last_peered":"2024-04-17T06:10:27.784233+0000","last_clean":"2024-04-17T06:10:27.784233+0000","last_became_active":"2024-04-17T06:10:27.777426+0000","last_became_peered":"2024-04-17T06:10:27.777426+0000","last_unstale":"2024-04-17T06:10:27.784233+0000","last_undegraded":"2024-04-17T06:10:27.784233+0000","last_fullsized":"2024-04-17T06:10:27.784233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:56.558792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797339+0000","last_change":"2024-04-17T06:10:27.791689+0000","last_active":"2024-04-17T06:10:27.797339+0000","last_peered":"2024-04-17T06:10:27.797339+0000","last_clean":"2024-04-17T06:10:27.797339+0000","last_became_active":"2024-04-17T06:10:27.791266+0000","last_became_peered":"2024-04-17T06:10:27.791266+0000","last_unstale":"2024-04-17T06:10:27.797339+0000","last_undegraded":"2024-04-17T06:10:27.797339+0000","last_fullsized":"2024-04-17T06:10:27.797339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:08.364826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.785348+0000","last_change":"2024-04-17T06:10:27.781355+0000","last_active":"2024-04-17T06:10:27.785348+0000","last_peered":"2024-04-17T06:10:27.785348+0000","last_clean":"2024-04-17T06:10:27.785348+0000","last_became_active":"2024-04-17T06:10:27.779488+0000","last_became_peered":"2024-04-17T06:10:27.779488+0000","last_unstale":"2024-04-17T06:10:27.785348+0000","last_undegraded":"2024-04-17T06:10:27.785348+0000","last_fullsized":"2024-04-17T06:10:27.785348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:59.939671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.802376+0000","last_change":"2024-04-17T06:10:27.797855+0000","last_active":"2024-04-17T06:10:27.802376+0000","last_peered":"2024-04-17T06:10:27.802376+0000","last_clean":"2024-04-17T06:10:27.802376+0000","last_became_active":"2024-04-17T06:10:27.797540+0000","last_became_peered":"2024-04-17T06:10:27.797540+0000","last_unstale":"2024-04-17T06:10:27.802376+0000","last_undegraded":"2024-04-17T06:10:27.802376+0000","last_fullsized":"2024-04-17T06:10:27.802376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:53.568351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.785133+0000","last_change":"2024-04-17T06:10:27.782568+0000","last_active":"2024-04-17T06:10:27.785133+0000","last_peered":"2024-04-17T06:10:27.785133+0000","last_clean":"2024-04-17T06:10:27.785133+0000","last_became_active":"2024-04-17T06:10:27.782308+0000","last_became_peered":"2024-04-17T06:10:27.782308+0000","last_unstale":"2024-04-17T06:10:27.785133+0000","last_undegraded":"2024-04-17T06:10:27.785133+0000","last_fullsized":"2024-04-17T06:10:27.785133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:30.540315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798927+0000","last_change":"2024-04-17T06:10:27.795904+0000","last_active":"2024-04-17T06:10:27.798927+0000","last_peered":"2024-04-17T06:10:27.798927+0000","last_clean":"2024-04-17T06:10:27.798927+0000","last_became_active":"2024-04-17T06:10:27.795629+0000","last_became_peered":"2024-04-17T06:10:27.795629+0000","last_unstale":"2024-04-17T06:10:27.798927+0000","last_undegraded":"2024-04-17T06:10:27.798927+0000","last_fullsized":"2024-04-17T06:10:27.798927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798079+0000","last_change":"2024-04-17T06:10:27.794083+0000","last_active":"2024-04-17T06:10:27.798079+0000","last_peered":"2024-04-17T06:10:27.798079+0000","last_clean":"2024-04-17T06:10:27.798079+0000","last_became_active":"2024-04-17T06:10:27.793777+0000","last_became_peered":"2024-04-17T06:10:27.793777+0000","last_unstale":"2024-04-17T06:10:27.798079+0000","last_undegraded":"2024-04-17T06:10:27.798079+0000","last_fullsized":"2024-04-17T06:10:27.798079+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:54.076686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797476+0000","last_change":"2024-04-17T06:10:27.791560+0000","last_active":"2024-04-17T06:10:27.797476+0000","last_peered":"2024-04-17T06:10:27.797476+0000","last_clean":"2024-04-17T06:10:27.797476+0000","last_became_active":"2024-04-17T06:10:27.791277+0000","last_became_peered":"2024-04-17T06:10:27.791277+0000","last_unstale":"2024-04-17T06:10:27.797476+0000","last_undegraded":"2024-04-17T06:10:27.797476+0000","last_fullsized":"2024-04-17T06:10:27.797476+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:13.072810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797380+0000","last_change":"2024-04-17T06:10:27.788991+0000","last_active":"2024-04-17T06:10:27.797380+0000","last_peered":"2024-04-17T06:10:27.797380+0000","last_clean":"2024-04-17T06:10:27.797380+0000","last_became_active":"2024-04-17T06:10:27.788251+0000","last_became_peered":"2024-04-17T06:10:27.788251+0000","last_unstale":"2024-04-17T06:10:27.797380+0000","last_undegraded":"2024-04-17T06:10:27.797380+0000","last_fullsized":"2024-04-17T06:10:27.797380+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:38.108803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796679+0000","last_change":"2024-04-17T06:10:26.784120+0000","last_active":"2024-04-17T06:10:27.796679+0000","last_peered":"2024-04-17T06:10:27.796679+0000","last_clean":"2024-04-17T06:10:27.796679+0000","last_became_active":"2024-04-17T06:10:26.783847+0000","last_became_peered":"2024-04-17T06:10:26.783847+0000","last_unstale":"2024-04-17T06:10:27.796679+0000","last_undegraded":"2024-04-17T06:10:27.796679+0000","last_fullsized":"2024-04-17T06:10:27.796679+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:08.112354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797385+0000","last_change":"2024-04-17T06:10:27.791051+0000","last_active":"2024-04-17T06:10:27.797385+0000","last_peered":"2024-04-17T06:10:27.797385+0000","last_clean":"2024-04-17T06:10:27.797385+0000","last_became_active":"2024-04-17T06:10:27.790781+0000","last_became_peered":"2024-04-17T06:10:27.790781+0000","last_unstale":"2024-04-17T06:10:27.797385+0000","last_undegraded":"2024-04-17T06:10:27.797385+0000","last_fullsized":"2024-04-17T06:10:27.797385+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:27.405253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792170+0000","last_change":"2024-04-17T06:10:26.787008+0000","last_active":"2024-04-17T06:10:27.792170+0000","last_peered":"2024-04-17T06:10:27.792170+0000","last_clean":"2024-04-17T06:10:27.792170+0000","last_became_active":"2024-04-17T06:10:26.786742+0000","last_became_peered":"2024-04-17T06:10:26.786742+0000","last_unstale":"2024-04-17T06:10:27.792170+0000","last_undegraded":"2024-04-17T06:10:27.792170+0000","last_fullsized":"2024-04-17T06:10:27.792170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:58.935115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.795660+0000","last_change":"2024-04-17T06:10:27.787055+0000","last_active":"2024-04-17T06:10:27.795660+0000","last_peered":"2024-04-17T06:10:27.795660+0000","last_clean":"2024-04-17T06:10:27.795660+0000","last_became_active":"2024-04-17T06:10:27.786821+0000","last_became_peered":"2024-04-17T06:10:27.786821+0000","last_unstale":"2024-04-17T06:10:27.795660+0000","last_undegraded":"2024-04-17T06:10:27.795660+0000","last_fullsized":"2024-04-17T06:10:27.795660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796683+0000","last_change":"2024-04-17T06:10:26.777762+0000","last_active":"2024-04-17T06:10:27.796683+0000","last_peered":"2024-04-17T06:10:27.796683+0000","last_clean":"2024-04-17T06:10:27.796683+0000","last_became_active":"2024-04-17T06:10:26.777522+0000","last_became_peered":"2024-04-17T06:10:26.777522+0000","last_unstale":"2024-04-17T06:10:27.796683+0000","last_undegraded":"2024-04-17T06:10:27.796683+0000","last_fullsized":"2024-04-17T06:10:27.796683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:47.393482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797932+0000","last_change":"2024-04-17T06:10:27.793308+0000","last_active":"2024-04-17T06:10:27.797932+0000","last_peered":"2024-04-17T06:10:27.797932+0000","last_clean":"2024-04-17T06:10:27.797932+0000","last_became_active":"2024-04-17T06:10:27.792964+0000","last_became_peered":"2024-04-17T06:10:27.792964+0000","last_unstale":"2024-04-17T06:10:27.797932+0000","last_undegraded":"2024-04-17T06:10:27.797932+0000","last_fullsized":"2024-04-17T06:10:27.797932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:06.115386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797359+0000","last_change":"2024-04-17T06:10:26.785844+0000","last_active":"2024-04-17T06:10:27.797359+0000","last_peered":"2024-04-17T06:10:27.797359+0000","last_clean":"2024-04-17T06:10:27.797359+0000","last_became_active":"2024-04-17T06:10:26.785544+0000","last_became_peered":"2024-04-17T06:10:26.785544+0000","last_unstale":"2024-04-17T06:10:27.797359+0000","last_undegraded":"2024-04-17T06:10:27.797359+0000","last_fullsized":"2024-04-17T06:10:27.797359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:58.935115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792428+0000","last_change":"2024-04-17T06:10:27.784936+0000","last_active":"2024-04-17T06:10:27.792428+0000","last_peered":"2024-04-17T06:10:27.792428+0000","last_clean":"2024-04-17T06:10:27.792428+0000","last_became_active":"2024-04-17T06:10:27.784624+0000","last_became_peered":"2024-04-17T06:10:27.784624+0000","last_unstale":"2024-04-17T06:10:27.792428+0000","last_undegraded":"2024-04-17T06:10:27.792428+0000","last_fullsized":"2024-04-17T06:10:27.792428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:48.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797885+0000","last_change":"2024-04-17T06:10:26.786620+0000","last_active":"2024-04-17T06:10:27.797885+0000","last_peered":"2024-04-17T06:10:27.797885+0000","last_clean":"2024-04-17T06:10:27.797885+0000","last_became_active":"2024-04-17T06:10:26.786331+0000","last_became_peered":"2024-04-17T06:10:26.786331+0000","last_unstale":"2024-04-17T06:10:27.797885+0000","last_undegraded":"2024-04-17T06:10:27.797885+0000","last_fullsized":"2024-04-17T06:10:27.797885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:37.104248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.786305+0000","last_change":"2024-04-17T06:10:27.782962+0000","last_active":"2024-04-17T06:10:27.786305+0000","last_peered":"2024-04-17T06:10:27.786305+0000","last_clean":"2024-04-17T06:10:27.786305+0000","last_became_active":"2024-04-17T06:10:27.782670+0000","last_became_peered":"2024-04-17T06:10:27.782670+0000","last_unstale":"2024-04-17T06:10:27.786305+0000","last_undegraded":"2024-04-17T06:10:27.786305+0000","last_fullsized":"2024-04-17T06:10:27.786305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:32:24.474707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.799747+0000","last_change":"2024-04-17T06:10:26.782183+0000","last_active":"2024-04-17T06:10:27.799747+0000","last_peered":"2024-04-17T06:10:27.799747+0000","last_clean":"2024-04-17T06:10:27.799747+0000","last_became_active":"2024-04-17T06:10:26.781885+0000","last_became_peered":"2024-04-17T06:10:26.781885+0000","last_unstale":"2024-04-17T06:10:27.799747+0000","last_undegraded":"2024-04-17T06:10:27.799747+0000","last_fullsized":"2024-04-17T06:10:27.799747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:58.935115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.795935+0000","last_change":"2024-04-17T06:10:27.786328+0000","last_active":"2024-04-17T06:10:27.795935+0000","last_peered":"2024-04-17T06:10:27.795935+0000","last_clean":"2024-04-17T06:10:27.795935+0000","last_became_active":"2024-04-17T06:10:27.786049+0000","last_became_peered":"2024-04-17T06:10:27.786049+0000","last_unstale":"2024-04-17T06:10:27.795935+0000","last_undegraded":"2024-04-17T06:10:27.795935+0000","last_fullsized":"2024-04-17T06:10:27.795935+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:48.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.781571+0000","last_change":"2024-04-17T06:10:26.782942+0000","last_active":"2024-04-17T06:10:27.781571+0000","last_peered":"2024-04-17T06:10:27.781571+0000","last_clean":"2024-04-17T06:10:27.781571+0000","last_became_active":"2024-04-17T06:10:26.782661+0000","last_became_peered":"2024-04-17T06:10:26.782661+0000","last_unstale":"2024-04-17T06:10:27.781571+0000","last_undegraded":"2024-04-17T06:10:27.781571+0000","last_fullsized":"2024-04-17T06:10:27.781571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:58.935115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798127+0000","last_change":"2024-04-17T06:10:27.794238+0000","last_active":"2024-04-17T06:10:27.798127+0000","last_peered":"2024-04-17T06:10:27.798127+0000","last_clean":"2024-04-17T06:10:27.798127+0000","last_became_active":"2024-04-17T06:10:27.793714+0000","last_became_peered":"2024-04-17T06:10:27.793714+0000","last_unstale":"2024-04-17T06:10:27.798127+0000","last_undegraded":"2024-04-17T06:10:27.798127+0000","last_fullsized":"2024-04-17T06:10:27.798127+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:32.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.775954+0000","last_change":"2024-04-17T06:10:26.780773+0000","last_active":"2024-04-17T06:10:27.775954+0000","last_peered":"2024-04-17T06:10:27.775954+0000","last_clean":"2024-04-17T06:10:27.775954+0000","last_became_active":"2024-04-17T06:10:26.780489+0000","last_became_peered":"2024-04-17T06:10:26.780489+0000","last_unstale":"2024-04-17T06:10:27.775954+0000","last_undegraded":"2024-04-17T06:10:27.775954+0000","last_fullsized":"2024-04-17T06:10:27.775954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:08.112354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.789721+0000","last_change":"2024-04-17T06:10:27.786059+0000","last_active":"2024-04-17T06:10:27.789721+0000","last_peered":"2024-04-17T06:10:27.789721+0000","last_clean":"2024-04-17T06:10:27.789721+0000","last_became_active":"2024-04-17T06:10:27.785780+0000","last_became_peered":"2024-04-17T06:10:27.785780+0000","last_unstale":"2024-04-17T06:10:27.789721+0000","last_undegraded":"2024-04-17T06:10:27.789721+0000","last_fullsized":"2024-04-17T06:10:27.789721+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:34.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.784030+0000","last_change":"2024-04-17T06:10:26.784585+0000","last_active":"2024-04-17T06:10:27.784030+0000","last_peered":"2024-04-17T06:10:27.784030+0000","last_clean":"2024-04-17T06:10:27.784030+0000","last_became_active":"2024-04-17T06:10:26.784337+0000","last_became_peered":"2024-04-17T06:10:26.784337+0000","last_unstale":"2024-04-17T06:10:27.784030+0000","last_undegraded":"2024-04-17T06:10:27.784030+0000","last_fullsized":"2024-04-17T06:10:27.784030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:20.097285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797855+0000","last_change":"2024-04-17T06:10:27.790494+0000","last_active":"2024-04-17T06:10:27.797855+0000","last_peered":"2024-04-17T06:10:27.797855+0000","last_clean":"2024-04-17T06:10:27.797855+0000","last_became_active":"2024-04-17T06:10:27.790229+0000","last_became_peered":"2024-04-17T06:10:27.790229+0000","last_unstale":"2024-04-17T06:10:27.797855+0000","last_undegraded":"2024-04-17T06:10:27.797855+0000","last_fullsized":"2024-04-17T06:10:27.797855+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:37.229605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.770480+0000","last_change":"2024-04-17T06:10:26.785233+0000","last_active":"2024-04-17T06:10:27.770480+0000","last_peered":"2024-04-17T06:10:27.770480+0000","last_clean":"2024-04-17T06:10:27.770480+0000","last_became_active":"2024-04-17T06:10:26.784966+0000","last_became_peered":"2024-04-17T06:10:26.784966+0000","last_unstale":"2024-04-17T06:10:27.770480+0000","last_undegraded":"2024-04-17T06:10:27.770480+0000","last_fullsized":"2024-04-17T06:10:27.770480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:22.561927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792387+0000","last_change":"2024-04-17T06:10:27.787597+0000","last_active":"2024-04-17T06:10:27.792387+0000","last_peered":"2024-04-17T06:10:27.792387+0000","last_clean":"2024-04-17T06:10:27.792387+0000","last_became_active":"2024-04-17T06:10:27.787247+0000","last_became_peered":"2024-04-17T06:10:27.787247+0000","last_unstale":"2024-04-17T06:10:27.792387+0000","last_undegraded":"2024-04-17T06:10:27.792387+0000","last_fullsized":"2024-04-17T06:10:27.792387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:08.364826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798975+0000","last_change":"2024-04-17T06:10:26.784372+0000","last_active":"2024-04-17T06:10:27.798975+0000","last_peered":"2024-04-17T06:10:27.798975+0000","last_clean":"2024-04-17T06:10:27.798975+0000","last_became_active":"2024-04-17T06:10:26.784105+0000","last_became_peered":"2024-04-17T06:10:26.784105+0000","last_unstale":"2024-04-17T06:10:27.798975+0000","last_undegraded":"2024-04-17T06:10:27.798975+0000","last_fullsized":"2024-04-17T06:10:27.798975+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:50.424352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.799359+0000","last_change":"2024-04-17T06:10:27.792824+0000","last_active":"2024-04-17T06:10:27.799359+0000","last_peered":"2024-04-17T06:10:27.799359+0000","last_clean":"2024-04-17T06:10:27.799359+0000","last_became_active":"2024-04-17T06:10:27.792546+0000","last_became_peered":"2024-04-17T06:10:27.792546+0000","last_unstale":"2024-04-17T06:10:27.799359+0000","last_undegraded":"2024-04-17T06:10:27.799359+0000","last_fullsized":"2024-04-17T06:10:27.799359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:13.072810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.805165+0000","last_change":"2024-04-17T06:10:26.788487+0000","last_active":"2024-04-17T06:10:27.805165+0000","last_peered":"2024-04-17T06:10:27.805165+0000","last_clean":"2024-04-17T06:10:27.805165+0000","last_became_active":"2024-04-17T06:10:26.787675+0000","last_became_peered":"2024-04-17T06:10:26.787675+0000","last_unstale":"2024-04-17T06:10:27.805165+0000","last_undegraded":"2024-04-17T06:10:27.805165+0000","last_fullsized":"2024-04-17T06:10:27.805165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:57.765286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792447+0000","last_change":"2024-04-17T06:10:27.785528+0000","last_active":"2024-04-17T06:10:27.792447+0000","last_peered":"2024-04-17T06:10:27.792447+0000","last_clean":"2024-04-17T06:10:27.792447+0000","last_became_active":"2024-04-17T06:10:27.785276+0000","last_became_peered":"2024-04-17T06:10:27.785276+0000","last_unstale":"2024-04-17T06:10:27.792447+0000","last_undegraded":"2024-04-17T06:10:27.792447+0000","last_fullsized":"2024-04-17T06:10:27.792447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.532545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.793278+0000","last_change":"2024-04-17T06:10:26.788770+0000","last_active":"2024-04-17T06:10:27.793278+0000","last_peered":"2024-04-17T06:10:27.793278+0000","last_clean":"2024-04-17T06:10:27.793278+0000","last_became_active":"2024-04-17T06:10:26.788541+0000","last_became_peered":"2024-04-17T06:10:26.788541+0000","last_unstale":"2024-04-17T06:10:27.793278+0000","last_undegraded":"2024-04-17T06:10:27.793278+0000","last_fullsized":"2024-04-17T06:10:27.793278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:32:23.470152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798140+0000","last_change":"2024-04-17T06:10:27.793976+0000","last_active":"2024-04-17T06:10:27.798140+0000","last_peered":"2024-04-17T06:10:27.798140+0000","last_clean":"2024-04-17T06:10:27.798140+0000","last_became_active":"2024-04-17T06:10:27.793159+0000","last_became_peered":"2024-04-17T06:10:27.793159+0000","last_unstale":"2024-04-17T06:10:27.798140+0000","last_undegraded":"2024-04-17T06:10:27.798140+0000","last_fullsized":"2024-04-17T06:10:27.798140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:53.568351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.781848+0000","last_change":"2024-04-17T06:10:26.782356+0000","last_active":"2024-04-17T06:10:27.781848+0000","last_peered":"2024-04-17T06:10:27.781848+0000","last_clean":"2024-04-17T06:10:27.781848+0000","last_became_active":"2024-04-17T06:10:26.782011+0000","last_became_peered":"2024-04-17T06:10:26.782011+0000","last_unstale":"2024-04-17T06:10:27.781848+0000","last_undegraded":"2024-04-17T06:10:27.781848+0000","last_fullsized":"2024-04-17T06:10:27.781848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:15.527989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797375+0000","last_change":"2024-04-17T06:10:27.791932+0000","last_active":"2024-04-17T06:10:27.797375+0000","last_peered":"2024-04-17T06:10:27.797375+0000","last_clean":"2024-04-17T06:10:27.797375+0000","last_became_active":"2024-04-17T06:10:27.791662+0000","last_became_peered":"2024-04-17T06:10:27.791662+0000","last_unstale":"2024-04-17T06:10:27.797375+0000","last_undegraded":"2024-04-17T06:10:27.797375+0000","last_fullsized":"2024-04-17T06:10:27.797375+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:08.364826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.769813+0000","last_change":"2024-04-17T06:10:26.784025+0000","last_active":"2024-04-17T06:10:27.769813+0000","last_peered":"2024-04-17T06:10:27.769813+0000","last_clean":"2024-04-17T06:10:27.769813+0000","last_became_active":"2024-04-17T06:10:26.783742+0000","last_became_peered":"2024-04-17T06:10:26.783742+0000","last_unstale":"2024-04-17T06:10:27.769813+0000","last_undegraded":"2024-04-17T06:10:27.769813+0000","last_fullsized":"2024-04-17T06:10:27.769813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:51:40.914371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798106+0000","last_change":"2024-04-17T06:10:27.794031+0000","last_active":"2024-04-17T06:10:27.798106+0000","last_peered":"2024-04-17T06:10:27.798106+0000","last_clean":"2024-04-17T06:10:27.798106+0000","last_became_active":"2024-04-17T06:10:27.793683+0000","last_became_peered":"2024-04-17T06:10:27.793683+0000","last_unstale":"2024-04-17T06:10:27.798106+0000","last_undegraded":"2024-04-17T06:10:27.798106+0000","last_fullsized":"2024-04-17T06:10:27.798106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:13.408047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.803406+0000","last_change":"2024-04-17T06:10:26.781122+0000","last_active":"2024-04-17T06:10:27.803406+0000","last_peered":"2024-04-17T06:10:27.803406+0000","last_clean":"2024-04-17T06:10:27.803406+0000","last_became_active":"2024-04-17T06:10:26.780824+0000","last_became_peered":"2024-04-17T06:10:26.780824+0000","last_unstale":"2024-04-17T06:10:27.803406+0000","last_undegraded":"2024-04-17T06:10:27.803406+0000","last_fullsized":"2024-04-17T06:10:27.803406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:15.527989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797765+0000","last_change":"2024-04-17T06:10:27.793828+0000","last_active":"2024-04-17T06:10:27.797765+0000","last_peered":"2024-04-17T06:10:27.797765+0000","last_clean":"2024-04-17T06:10:27.797765+0000","last_became_active":"2024-04-17T06:10:27.793501+0000","last_became_peered":"2024-04-17T06:10:27.793501+0000","last_unstale":"2024-04-17T06:10:27.797765+0000","last_undegraded":"2024-04-17T06:10:27.797765+0000","last_fullsized":"2024-04-17T06:10:27.797765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:48.399109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.802740+0000","last_change":"2024-04-17T06:10:26.782705+0000","last_active":"2024-04-17T06:10:27.802740+0000","last_peered":"2024-04-17T06:10:27.802740+0000","last_clean":"2024-04-17T06:10:27.802740+0000","last_became_active":"2024-04-17T06:10:26.781684+0000","last_became_peered":"2024-04-17T06:10:26.781684+0000","last_unstale":"2024-04-17T06:10:27.802740+0000","last_undegraded":"2024-04-17T06:10:27.802740+0000","last_fullsized":"2024-04-17T06:10:27.802740+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:53.072130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.786011+0000","last_change":"2024-04-17T06:10:27.782130+0000","last_active":"2024-04-17T06:10:27.786011+0000","last_peered":"2024-04-17T06:10:27.786011+0000","last_clean":"2024-04-17T06:10:27.786011+0000","last_became_active":"2024-04-17T06:10:27.781825+0000","last_became_peered":"2024-04-17T06:10:27.781825+0000","last_unstale":"2024-04-17T06:10:27.786011+0000","last_undegraded":"2024-04-17T06:10:27.786011+0000","last_fullsized":"2024-04-17T06:10:27.786011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:21.101840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792343+0000","last_change":"2024-04-17T06:10:26.779973+0000","last_active":"2024-04-17T06:10:27.792343+0000","last_peered":"2024-04-17T06:10:27.792343+0000","last_clean":"2024-04-17T06:10:27.792343+0000","last_became_active":"2024-04-17T06:10:26.779656+0000","last_became_peered":"2024-04-17T06:10:26.779656+0000","last_unstale":"2024-04-17T06:10:27.792343+0000","last_undegraded":"2024-04-17T06:10:27.792343+0000","last_fullsized":"2024-04-17T06:10:27.792343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:31.670808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.795759+0000","last_change":"2024-04-17T06:10:27.790149+0000","last_active":"2024-04-17T06:10:27.795759+0000","last_peered":"2024-04-17T06:10:27.795759+0000","last_clean":"2024-04-17T06:10:27.795759+0000","last_became_active":"2024-04-17T06:10:27.787463+0000","last_became_peered":"2024-04-17T06:10:27.787463+0000","last_unstale":"2024-04-17T06:10:27.795759+0000","last_undegraded":"2024-04-17T06:10:27.795759+0000","last_fullsized":"2024-04-17T06:10:27.795759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:27.405253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.789898+0000","last_change":"2024-04-17T06:10:26.779273+0000","last_active":"2024-04-17T06:10:27.789898+0000","last_peered":"2024-04-17T06:10:27.789898+0000","last_clean":"2024-04-17T06:10:27.789898+0000","last_became_active":"2024-04-17T06:10:26.779026+0000","last_became_peered":"2024-04-17T06:10:26.779026+0000","last_unstale":"2024-04-17T06:10:27.789898+0000","last_undegraded":"2024-04-17T06:10:27.789898+0000","last_fullsized":"2024-04-17T06:10:27.789898+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:33.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798398+0000","last_change":"2024-04-17T06:10:27.794013+0000","last_active":"2024-04-17T06:10:27.798398+0000","last_peered":"2024-04-17T06:10:27.798398+0000","last_clean":"2024-04-17T06:10:27.798398+0000","last_became_active":"2024-04-17T06:10:27.793756+0000","last_became_peered":"2024-04-17T06:10:27.793756+0000","last_unstale":"2024-04-17T06:10:27.798398+0000","last_undegraded":"2024-04-17T06:10:27.798398+0000","last_fullsized":"2024-04-17T06:10:27.798398+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:30.129352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.781597+0000","last_change":"2024-04-17T06:10:26.778012+0000","last_active":"2024-04-17T06:10:27.781597+0000","last_peered":"2024-04-17T06:10:27.781597+0000","last_clean":"2024-04-17T06:10:27.781597+0000","last_became_active":"2024-04-17T06:10:26.777094+0000","last_became_peered":"2024-04-17T06:10:26.777094+0000","last_unstale":"2024-04-17T06:10:27.781597+0000","last_undegraded":"2024-04-17T06:10:27.781597+0000","last_fullsized":"2024-04-17T06:10:27.781597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:29.535760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798117+0000","last_change":"2024-04-17T06:10:27.794492+0000","last_active":"2024-04-17T06:10:27.798117+0000","last_peered":"2024-04-17T06:10:27.798117+0000","last_clean":"2024-04-17T06:10:27.798117+0000","last_became_active":"2024-04-17T06:10:27.794290+0000","last_became_peered":"2024-04-17T06:10:27.794290+0000","last_unstale":"2024-04-17T06:10:27.798117+0000","last_undegraded":"2024-04-17T06:10:27.798117+0000","last_fullsized":"2024-04-17T06:10:27.798117+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:11.621227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.790175+0000","last_change":"2024-04-17T06:10:26.787665+0000","last_active":"2024-04-17T06:10:27.790175+0000","last_peered":"2024-04-17T06:10:27.790175+0000","last_clean":"2024-04-17T06:10:27.790175+0000","last_became_active":"2024-04-17T06:10:26.787080+0000","last_became_peered":"2024-04-17T06:10:26.787080+0000","last_unstale":"2024-04-17T06:10:27.790175+0000","last_undegraded":"2024-04-17T06:10:27.790175+0000","last_fullsized":"2024-04-17T06:10:27.790175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:45.709078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792662+0000","last_change":"2024-04-17T06:10:27.787597+0000","last_active":"2024-04-17T06:10:27.792662+0000","last_peered":"2024-04-17T06:10:27.792662+0000","last_clean":"2024-04-17T06:10:27.792662+0000","last_became_active":"2024-04-17T06:10:27.787255+0000","last_became_peered":"2024-04-17T06:10:27.787255+0000","last_unstale":"2024-04-17T06:10:27.792662+0000","last_undegraded":"2024-04-17T06:10:27.792662+0000","last_fullsized":"2024-04-17T06:10:27.792662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:13.072810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.771128+0000","last_change":"2024-04-17T06:10:26.782206+0000","last_active":"2024-04-17T06:10:27.771128+0000","last_peered":"2024-04-17T06:10:27.771128+0000","last_clean":"2024-04-17T06:10:27.771128+0000","last_became_active":"2024-04-17T06:10:26.781863+0000","last_became_peered":"2024-04-17T06:10:26.781863+0000","last_unstale":"2024-04-17T06:10:27.771128+0000","last_undegraded":"2024-04-17T06:10:27.771128+0000","last_fullsized":"2024-04-17T06:10:27.771128+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:43:11.146601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.776349+0000","last_change":"2024-04-17T06:10:27.774508+0000","last_active":"2024-04-17T06:10:27.776349+0000","last_peered":"2024-04-17T06:10:27.776349+0000","last_clean":"2024-04-17T06:10:27.776349+0000","last_became_active":"2024-04-17T06:10:27.774209+0000","last_became_peered":"2024-04-17T06:10:27.774209+0000","last_unstale":"2024-04-17T06:10:27.776349+0000","last_undegraded":"2024-04-17T06:10:27.776349+0000","last_fullsized":"2024-04-17T06:10:27.776349+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:37.229605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.777174+0000","last_change":"2024-04-17T06:10:26.780639+0000","last_active":"2024-04-17T06:10:27.777174+0000","last_peered":"2024-04-17T06:10:27.777174+0000","last_clean":"2024-04-17T06:10:27.777174+0000","last_became_active":"2024-04-17T06:10:26.780362+0000","last_became_peered":"2024-04-17T06:10:26.780362+0000","last_unstale":"2024-04-17T06:10:27.777174+0000","last_undegraded":"2024-04-17T06:10:27.777174+0000","last_fullsized":"2024-04-17T06:10:27.777174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:47.393482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798145+0000","last_change":"2024-04-17T06:10:27.794238+0000","last_active":"2024-04-17T06:10:27.798145+0000","last_peered":"2024-04-17T06:10:27.798145+0000","last_clean":"2024-04-17T06:10:27.798145+0000","last_became_active":"2024-04-17T06:10:27.793955+0000","last_became_peered":"2024-04-17T06:10:27.793955+0000","last_unstale":"2024-04-17T06:10:27.798145+0000","last_undegraded":"2024-04-17T06:10:27.798145+0000","last_fullsized":"2024-04-17T06:10:27.798145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:56.939325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.770970+0000","last_change":"2024-04-17T06:10:26.786485+0000","last_active":"2024-04-17T06:10:27.770970+0000","last_peered":"2024-04-17T06:10:27.770970+0000","last_clean":"2024-04-17T06:10:27.770970+0000","last_became_active":"2024-04-17T06:10:26.786185+0000","last_became_peered":"2024-04-17T06:10:26.786185+0000","last_unstale":"2024-04-17T06:10:27.770970+0000","last_undegraded":"2024-04-17T06:10:27.770970+0000","last_fullsized":"2024-04-17T06:10:27.770970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:12.998924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797360+0000","last_change":"2024-04-17T06:10:27.791808+0000","last_active":"2024-04-17T06:10:27.797360+0000","last_peered":"2024-04-17T06:10:27.797360+0000","last_clean":"2024-04-17T06:10:27.797360+0000","last_became_active":"2024-04-17T06:10:27.791518+0000","last_became_peered":"2024-04-17T06:10:27.791518+0000","last_unstale":"2024-04-17T06:10:27.797360+0000","last_undegraded":"2024-04-17T06:10:27.797360+0000","last_fullsized":"2024-04-17T06:10:27.797360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:23.566483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.799478+0000","last_change":"2024-04-17T06:10:26.785898+0000","last_active":"2024-04-17T06:10:27.799478+0000","last_peered":"2024-04-17T06:10:27.799478+0000","last_clean":"2024-04-17T06:10:27.799478+0000","last_became_active":"2024-04-17T06:10:26.785529+0000","last_became_peered":"2024-04-17T06:10:26.785529+0000","last_unstale":"2024-04-17T06:10:27.799478+0000","last_undegraded":"2024-04-17T06:10:27.799478+0000","last_fullsized":"2024-04-17T06:10:27.799478+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:10.616672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797240+0000","last_change":"2024-04-17T06:10:27.787649+0000","last_active":"2024-04-17T06:10:27.797240+0000","last_peered":"2024-04-17T06:10:27.797240+0000","last_clean":"2024-04-17T06:10:27.797240+0000","last_became_active":"2024-04-17T06:10:27.787354+0000","last_became_peered":"2024-04-17T06:10:27.787354+0000","last_unstale":"2024-04-17T06:10:27.797240+0000","last_undegraded":"2024-04-17T06:10:27.797240+0000","last_fullsized":"2024-04-17T06:10:27.797240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:27.405253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.771400+0000","last_change":"2024-04-17T06:10:26.781476+0000","last_active":"2024-04-17T06:10:27.771400+0000","last_peered":"2024-04-17T06:10:27.771400+0000","last_clean":"2024-04-17T06:10:27.771400+0000","last_became_active":"2024-04-17T06:10:26.781228+0000","last_became_peered":"2024-04-17T06:10:26.781228+0000","last_unstale":"2024-04-17T06:10:27.771400+0000","last_undegraded":"2024-04-17T06:10:27.771400+0000","last_fullsized":"2024-04-17T06:10:27.771400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:46.790174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.800215+0000","last_change":"2024-04-17T06:10:27.797688+0000","last_active":"2024-04-17T06:10:27.800215+0000","last_peered":"2024-04-17T06:10:27.800215+0000","last_clean":"2024-04-17T06:10:27.800215+0000","last_became_active":"2024-04-17T06:10:27.797379+0000","last_became_peered":"2024-04-17T06:10:27.797379+0000","last_unstale":"2024-04-17T06:10:27.800215+0000","last_undegraded":"2024-04-17T06:10:27.800215+0000","last_fullsized":"2024-04-17T06:10:27.800215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:28.568615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.776078+0000","last_change":"2024-04-17T06:10:26.791062+0000","last_active":"2024-04-17T06:10:27.776078+0000","last_peered":"2024-04-17T06:10:27.776078+0000","last_clean":"2024-04-17T06:10:27.776078+0000","last_became_active":"2024-04-17T06:10:26.790745+0000","last_became_peered":"2024-04-17T06:10:26.790745+0000","last_unstale":"2024-04-17T06:10:27.776078+0000","last_undegraded":"2024-04-17T06:10:27.776078+0000","last_fullsized":"2024-04-17T06:10:27.776078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:29.535760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798073+0000","last_change":"2024-04-17T06:10:27.793357+0000","last_active":"2024-04-17T06:10:27.798073+0000","last_peered":"2024-04-17T06:10:27.798073+0000","last_clean":"2024-04-17T06:10:27.798073+0000","last_became_active":"2024-04-17T06:10:27.792995+0000","last_became_peered":"2024-04-17T06:10:27.792995+0000","last_unstale":"2024-04-17T06:10:27.798073+0000","last_undegraded":"2024-04-17T06:10:27.798073+0000","last_fullsized":"2024-04-17T06:10:27.798073+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:51.428907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.774589+0000","last_change":"2024-04-17T06:10:26.786334+0000","last_active":"2024-04-17T06:10:27.774589+0000","last_peered":"2024-04-17T06:10:27.774589+0000","last_clean":"2024-04-17T06:10:27.774589+0000","last_became_active":"2024-04-17T06:10:26.785955+0000","last_became_peered":"2024-04-17T06:10:26.785955+0000","last_unstale":"2024-04-17T06:10:27.774589+0000","last_undegraded":"2024-04-17T06:10:27.774589+0000","last_fullsized":"2024-04-17T06:10:27.774589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:55.554236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792509+0000","last_change":"2024-04-17T06:10:27.787841+0000","last_active":"2024-04-17T06:10:27.792509+0000","last_peered":"2024-04-17T06:10:27.792509+0000","last_clean":"2024-04-17T06:10:27.792509+0000","last_became_active":"2024-04-17T06:10:27.787587+0000","last_became_peered":"2024-04-17T06:10:27.787587+0000","last_unstale":"2024-04-17T06:10:27.792509+0000","last_undegraded":"2024-04-17T06:10:27.792509+0000","last_fullsized":"2024-04-17T06:10:27.792509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.771092+0000","last_change":"2024-04-17T06:10:26.786793+0000","last_active":"2024-04-17T06:10:27.771092+0000","last_peered":"2024-04-17T06:10:27.771092+0000","last_clean":"2024-04-17T06:10:27.771092+0000","last_became_active":"2024-04-17T06:10:26.786124+0000","last_became_peered":"2024-04-17T06:10:26.786124+0000","last_unstale":"2024-04-17T06:10:27.771092+0000","last_undegraded":"2024-04-17T06:10:27.771092+0000","last_fullsized":"2024-04-17T06:10:27.771092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:31.670808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.802501+0000","last_change":"2024-04-17T06:10:27.797912+0000","last_active":"2024-04-17T06:10:27.802501+0000","last_peered":"2024-04-17T06:10:27.802501+0000","last_clean":"2024-04-17T06:10:27.802501+0000","last_became_active":"2024-04-17T06:10:27.797641+0000","last_became_peered":"2024-04-17T06:10:27.797641+0000","last_unstale":"2024-04-17T06:10:27.802501+0000","last_undegraded":"2024-04-17T06:10:27.802501+0000","last_fullsized":"2024-04-17T06:10:27.802501+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:08.364826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.777640+0000","last_change":"2024-04-17T06:10:26.785181+0000","last_active":"2024-04-17T06:10:27.777640+0000","last_peered":"2024-04-17T06:10:27.777640+0000","last_clean":"2024-04-17T06:10:27.777640+0000","last_became_active":"2024-04-17T06:10:26.784736+0000","last_became_peered":"2024-04-17T06:10:26.784736+0000","last_unstale":"2024-04-17T06:10:27.777640+0000","last_undegraded":"2024-04-17T06:10:27.777640+0000","last_fullsized":"2024-04-17T06:10:27.777640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:55.554236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796702+0000","last_change":"2024-04-17T06:10:27.786963+0000","last_active":"2024-04-17T06:10:27.796702+0000","last_peered":"2024-04-17T06:10:27.796702+0000","last_clean":"2024-04-17T06:10:27.796702+0000","last_became_active":"2024-04-17T06:10:27.786671+0000","last_became_peered":"2024-04-17T06:10:27.786671+0000","last_unstale":"2024-04-17T06:10:27.796702+0000","last_undegraded":"2024-04-17T06:10:27.796702+0000","last_fullsized":"2024-04-17T06:10:27.796702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.783621+0000","last_change":"2024-04-17T06:10:26.783829+0000","last_active":"2024-04-17T06:10:27.783621+0000","last_peered":"2024-04-17T06:10:27.783621+0000","last_clean":"2024-04-17T06:10:27.783621+0000","last_became_active":"2024-04-17T06:10:26.783581+0000","last_became_peered":"2024-04-17T06:10:26.783581+0000","last_unstale":"2024-04-17T06:10:27.783621+0000","last_undegraded":"2024-04-17T06:10:27.783621+0000","last_fullsized":"2024-04-17T06:10:27.783621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:12.372508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.794021+0000","last_change":"2024-04-17T06:10:27.791774+0000","last_active":"2024-04-17T06:10:27.794021+0000","last_peered":"2024-04-17T06:10:27.794021+0000","last_clean":"2024-04-17T06:10:27.794021+0000","last_became_active":"2024-04-17T06:10:27.791474+0000","last_became_peered":"2024-04-17T06:10:27.791474+0000","last_unstale":"2024-04-17T06:10:27.794021+0000","last_undegraded":"2024-04-17T06:10:27.794021+0000","last_fullsized":"2024-04-17T06:10:27.794021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:48.774295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.776067+0000","last_change":"2024-04-17T06:10:26.786068+0000","last_active":"2024-04-17T06:10:27.776067+0000","last_peered":"2024-04-17T06:10:27.776067+0000","last_clean":"2024-04-17T06:10:27.776067+0000","last_became_active":"2024-04-17T06:10:26.785723+0000","last_became_peered":"2024-04-17T06:10:26.785723+0000","last_unstale":"2024-04-17T06:10:27.776067+0000","last_undegraded":"2024-04-17T06:10:27.776067+0000","last_fullsized":"2024-04-17T06:10:27.776067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:29.535760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.777762+0000","last_change":"2024-04-17T06:09:09.067035+0000","last_active":"2024-04-17T06:10:27.777762+0000","last_peered":"2024-04-17T06:10:27.777762+0000","last_clean":"2024-04-17T06:10:27.777762+0000","last_became_active":"2024-04-17T06:09:09.066130+0000","last_became_peered":"2024-04-17T06:09:09.066130+0000","last_unstale":"2024-04-17T06:10:27.777762+0000","last_undegraded":"2024-04-17T06:10:27.777762+0000","last_fullsized":"2024-04-17T06:10:27.777762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_clean_scrub_stamp":"2024-04-17T06:09:07.958875+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:03.277836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T06:10:30.768 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-17T06:10:30.907 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.909+0000 7f370cbfa640 1 Processor -- start 2024-04-17T06:10:30.908 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.909+0000 7f370cbfa640 1 -- start start 2024-04-17T06:10:30.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.910+0000 7f370cbfa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708073750 0x7f37080b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.910+0000 7f370cbfa640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37080b75c0 con 0x7f37080cebf0 2024-04-17T06:10:30.911 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.910+0000 7f370cbfa640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37080cf080 con 0x7f3708144f70 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.910+0000 7f370cbfa640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f3708160070 con 0x7f3708073750 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f3705d9b640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f37080cebf0 0x7f3708073040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41888/0 (socket says 172.21.15.135:41888) 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f3705d9b640 1 -- 172.21.15.135:0/2954362243 learned_addr learned my addr 172.21.15.135:0/2954362243 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371826213 0 0) 0x7f37080b75c0 con 0x7f37080cebf0 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36f4003520 con 0x7f37080cebf0 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f3706d9d640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708073750 0x7f37080b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 809799452 0 0) 0x7f36f4003520 con 0x7f37080cebf0 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708073750 msgr2=0x7f37080b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708073750 0x7f37080b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 >> v1:172.21.15.190:6789/0 conn(0x7f3708144f70 legacy=0x7f370815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3708160350 con 0x7f37080cebf0 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f36f00019f0 con 0x7f37080cebf0 2024-04-17T06:10:30.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 >> v1:172.21.15.135:6789/0 conn(0x7f37080cebf0 legacy=0x7f3708073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708073750 0x7f36f40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4003f60 0x7f36f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36f4004d40 0x7f36f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3708160070 con 0x7f36f4004d40 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f36f400bb50 con 0x7f36f4003f60 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f36f400df60 con 0x7f3708073750 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f3706d9d640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36f4004d40 0x7f36f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f3705d9b640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4003f60 0x7f36f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.911+0000 7f370659c640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708073750 0x7f36f40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.912+0000 7f3706d9d640 1 -- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708073750 msgr2=0x7f36f40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.912+0000 7f3706d9d640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708073750 0x7f36f40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.912+0000 7f3706d9d640 1 -- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4003f60 msgr2=0x7f36f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:30.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.912+0000 7f3706d9d640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4003f60 0x7f36f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.912+0000 7f3706d9d640 1 -- 172.21.15.135:0/2954362243 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f4003520 con 0x7f36f4004d40 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.912+0000 7f3706d9d640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36f4004d40 0x7f36f40051c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009ac0 tx=0x7f36f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48da7440d8cdbb server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.912+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8039070 con 0x7f36f4004d40 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f802c6b0 con 0x7f36f4004d40 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f370559a640 1 -- 172.21.15.135:0/2954362243 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8009690 con 0x7f36f4004d40 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f370cbfa640 1 -- 172.21.15.135:0/2954362243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36f4004d40 msgr2=0x7f36f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f370cbfa640 1 --2- 172.21.15.135:0/2954362243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36f4004d40 0x7f36f40051c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009ac0 tx=0x7f36f4005830 comp rx=0 tx=0).stop 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f3706d9d640 1 -- 172.21.15.135:0/2954362243 reap_dead start 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f370cbfa640 1 -- 172.21.15.135:0/2954362243 shutdown_connections 2024-04-17T06:10:30.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f370cbfa640 1 -- 172.21.15.135:0/2954362243 >> 172.21.15.135:0/2954362243 conn(0x7f37080d15e0 msgr2=0x7f37080d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:30.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f370cbfa640 1 -- 172.21.15.135:0/2954362243 shutdown_connections 2024-04-17T06:10:30.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.913+0000 7f370cbfa640 1 -- 172.21.15.135:0/2954362243 wait complete. 2024-04-17T06:10:30.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f370cbfa640 1 Processor -- start 2024-04-17T06:10:30.917 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f370cbfa640 1 -- start start 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f370cbfa640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4004d40 0x7f37081341a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f370cbfa640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f370815acb0 0x7f370815b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f370cbfa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708131df0 0x7f3708132270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f370cbfa640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f370815fc10 con 0x7f370815acb0 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f370cbfa640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f37080ce3f0 con 0x7f36f4004d40 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f370cbfa640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f370815fd90 con 0x7f3708131df0 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f3705d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f370815acb0 0x7f370815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f3705d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f370815acb0 0x7f370815b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54786/0 (socket says 172.21.15.135:54786) 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.915+0000 7f3705d9b640 1 -- 172.21.15.135:0/2558077164 learned_addr learned my addr 172.21.15.135:0/2558077164 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f370659c640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4004d40 0x7f37081341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f3706d9d640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708131df0 0x7f3708132270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f3705d9b640 1 -- 172.21.15.135:0/2558077164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708131df0 msgr2=0x7f3708132270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f3705d9b640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708131df0 0x7f3708132270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f3705d9b640 1 -- 172.21.15.135:0/2558077164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4004d40 msgr2=0x7f37081341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f3705d9b640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4004d40 0x7f37081341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f3705d9b640 1 -- 172.21.15.135:0/2558077164 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3708130d50 con 0x7f370815acb0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f3705d9b640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f370815acb0 0x7f370815b130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f36f00079a0 tx=0x7f36f000a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f36feffd640 1 -- 172.21.15.135:0/2558077164 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f00133f0 con 0x7f370815acb0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f36feffd640 1 -- 172.21.15.135:0/2558077164 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f0018070 con 0x7f370815acb0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f36feffd640 1 -- 172.21.15.135:0/2558077164 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f00137d0 con 0x7f370815acb0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3708131030 con 0x7f370815acb0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.916+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081315f0 con 0x7f370815acb0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.917+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3708068a20 con 0x7f370815acb0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.918+0000 7f36feffd640 1 -- 172.21.15.135:0/2558077164 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36f0013b80 con 0x7f370815acb0 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.918+0000 7f36feffd640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f36e806c8f0 0x7f36e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:30.919 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.918+0000 7f370659c640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f36e806c8f0 0x7f36e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:30.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.918+0000 7f36feffd640 1 -- 172.21.15.135:0/2558077164 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f36f008c900 con 0x7f370815acb0 2024-04-17T06:10:30.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.919+0000 7f370659c640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f36e806c8f0 0x7f36e806edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37000035f0 tx=0x7f3700009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:30.920 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:30.923+0000 7f36feffd640 1 -- 172.21.15.135:0/2558077164 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f36f005ac30 con 0x7f370815acb0 2024-04-17T06:10:31.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.094+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 --> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37080b6e90 con 0x7f36e806c8f0 2024-04-17T06:10:31.097 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.100+0000 7f36feffd640 1 -- 172.21.15.135:0/2558077164 <== mgr.4113 v2:172.21.15.135:6832/3190814219 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251887 (secure 0 0 0) 0x7f37080b6e90 con 0x7f36e806c8f0 2024-04-17T06:10:31.100 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:31.101 INFO:teuthology.orchestra.run.smithi135.stderr:dumped all 2024-04-17T06:10:31.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.102+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f36e806c8f0 msgr2=0x7f36e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.102+0000 7f370cbfa640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f36e806c8f0 0x7f36e806edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37000035f0 tx=0x7f3700009040 comp rx=0 tx=0).stop 2024-04-17T06:10:31.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.102+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f370815acb0 msgr2=0x7f370815b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.102+0000 7f370cbfa640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f370815acb0 0x7f370815b130 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f36f00079a0 tx=0x7f36f000a730 comp rx=0 tx=0).stop 2024-04-17T06:10:31.103 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.103+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 shutdown_connections 2024-04-17T06:10:31.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.103+0000 7f370cbfa640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3708131df0 0x7f3708132270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.103+0000 7f370cbfa640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f36e806c8f0 0x7f36e806edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.103+0000 7f370cbfa640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f370815acb0 0x7f370815b130 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.103+0000 7f370cbfa640 1 --2- 172.21.15.135:0/2558077164 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f36f4004d40 0x7f37081341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.103+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 >> 172.21.15.135:0/2558077164 conn(0x7f37080d15e0 msgr2=0x7f370815bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:31.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.103+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 shutdown_connections 2024-04-17T06:10:31.104 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.103+0000 7f370cbfa640 1 -- 172.21.15.135:0/2558077164 wait complete. 2024-04-17T06:10:31.112 INFO:teuthology.orchestra.run.smithi135.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-17T06:10:29.951283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220584,"kb_used_data":4408,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754136,"statfs":{"total":773094113280,"available":772868235264,"internally_reserved":0,"allocated":4513792,"data_stored":2597261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.192974"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797951+0000","last_change":"2024-04-17T06:10:27.793433+0000","last_active":"2024-04-17T06:10:27.797951+0000","last_peered":"2024-04-17T06:10:27.797951+0000","last_clean":"2024-04-17T06:10:27.797951+0000","last_became_active":"2024-04-17T06:10:27.792612+0000","last_became_peered":"2024-04-17T06:10:27.792612+0000","last_unstale":"2024-04-17T06:10:27.797951+0000","last_undegraded":"2024-04-17T06:10:27.797951+0000","last_fullsized":"2024-04-17T06:10:27.797951+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:48.774295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797817+0000","last_change":"2024-04-17T06:10:27.793271+0000","last_active":"2024-04-17T06:10:27.797817+0000","last_peered":"2024-04-17T06:10:27.797817+0000","last_clean":"2024-04-17T06:10:27.797817+0000","last_became_active":"2024-04-17T06:10:27.792418+0000","last_became_peered":"2024-04-17T06:10:27.792418+0000","last_unstale":"2024-04-17T06:10:27.797817+0000","last_undegraded":"2024-04-17T06:10:27.797817+0000","last_fullsized":"2024-04-17T06:10:27.797817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:45.511079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792412+0000","last_change":"2024-04-17T06:10:27.788173+0000","last_active":"2024-04-17T06:10:27.792412+0000","last_peered":"2024-04-17T06:10:27.792412+0000","last_clean":"2024-04-17T06:10:27.792412+0000","last_became_active":"2024-04-17T06:10:27.787899+0000","last_became_peered":"2024-04-17T06:10:27.787899+0000","last_unstale":"2024-04-17T06:10:27.792412+0000","last_undegraded":"2024-04-17T06:10:27.792412+0000","last_fullsized":"2024-04-17T06:10:27.792412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:23.566483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.789742+0000","last_change":"2024-04-17T06:10:27.786687+0000","last_active":"2024-04-17T06:10:27.789742+0000","last_peered":"2024-04-17T06:10:27.789742+0000","last_clean":"2024-04-17T06:10:27.789742+0000","last_became_active":"2024-04-17T06:10:27.786431+0000","last_became_peered":"2024-04-17T06:10:27.786431+0000","last_unstale":"2024-04-17T06:10:27.789742+0000","last_undegraded":"2024-04-17T06:10:27.789742+0000","last_fullsized":"2024-04-17T06:10:27.789742+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:31:29.646387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792347+0000","last_change":"2024-04-17T06:10:27.788301+0000","last_active":"2024-04-17T06:10:27.792347+0000","last_peered":"2024-04-17T06:10:27.792347+0000","last_clean":"2024-04-17T06:10:27.792347+0000","last_became_active":"2024-04-17T06:10:27.788062+0000","last_became_peered":"2024-04-17T06:10:27.788062+0000","last_unstale":"2024-04-17T06:10:27.792347+0000","last_undegraded":"2024-04-17T06:10:27.792347+0000","last_fullsized":"2024-04-17T06:10:27.792347+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:24:51.392110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798664+0000","last_change":"2024-04-17T06:10:27.794854+0000","last_active":"2024-04-17T06:10:27.798664+0000","last_peered":"2024-04-17T06:10:27.798664+0000","last_clean":"2024-04-17T06:10:27.798664+0000","last_became_active":"2024-04-17T06:10:27.794595+0000","last_became_peered":"2024-04-17T06:10:27.794595+0000","last_unstale":"2024-04-17T06:10:27.798664+0000","last_undegraded":"2024-04-17T06:10:27.798664+0000","last_fullsized":"2024-04-17T06:10:27.798664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:51.428907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.802315+0000","last_change":"2024-04-17T06:10:27.796909+0000","last_active":"2024-04-17T06:10:27.802315+0000","last_peered":"2024-04-17T06:10:27.802315+0000","last_clean":"2024-04-17T06:10:27.802315+0000","last_became_active":"2024-04-17T06:10:27.794908+0000","last_became_peered":"2024-04-17T06:10:27.794908+0000","last_unstale":"2024-04-17T06:10:27.802315+0000","last_undegraded":"2024-04-17T06:10:27.802315+0000","last_fullsized":"2024-04-17T06:10:27.802315+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:48.399109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797556+0000","last_change":"2024-04-17T06:10:27.792851+0000","last_active":"2024-04-17T06:10:27.797556+0000","last_peered":"2024-04-17T06:10:27.797556+0000","last_clean":"2024-04-17T06:10:27.797556+0000","last_became_active":"2024-04-17T06:10:27.792557+0000","last_became_peered":"2024-04-17T06:10:27.792557+0000","last_unstale":"2024-04-17T06:10:27.797556+0000","last_undegraded":"2024-04-17T06:10:27.797556+0000","last_fullsized":"2024-04-17T06:10:27.797556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:22:41.057096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792393+0000","last_change":"2024-04-17T06:10:27.786708+0000","last_active":"2024-04-17T06:10:27.792393+0000","last_peered":"2024-04-17T06:10:27.792393+0000","last_clean":"2024-04-17T06:10:27.792393+0000","last_became_active":"2024-04-17T06:10:27.786475+0000","last_became_peered":"2024-04-17T06:10:27.786475+0000","last_unstale":"2024-04-17T06:10:27.792393+0000","last_undegraded":"2024-04-17T06:10:27.792393+0000","last_fullsized":"2024-04-17T06:10:27.792393+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:27.405253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792783+0000","last_change":"2024-04-17T06:10:27.789058+0000","last_active":"2024-04-17T06:10:27.792783+0000","last_peered":"2024-04-17T06:10:27.792783+0000","last_clean":"2024-04-17T06:10:27.792783+0000","last_became_active":"2024-04-17T06:10:27.788774+0000","last_became_peered":"2024-04-17T06:10:27.788774+0000","last_unstale":"2024-04-17T06:10:27.792783+0000","last_undegraded":"2024-04-17T06:10:27.792783+0000","last_fullsized":"2024-04-17T06:10:27.792783+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:52:45.511079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.790507+0000","last_change":"2024-04-17T06:10:27.787093+0000","last_active":"2024-04-17T06:10:27.790507+0000","last_peered":"2024-04-17T06:10:27.790507+0000","last_clean":"2024-04-17T06:10:27.790507+0000","last_became_active":"2024-04-17T06:10:27.786834+0000","last_became_peered":"2024-04-17T06:10:27.786834+0000","last_unstale":"2024-04-17T06:10:27.790507+0000","last_undegraded":"2024-04-17T06:10:27.790507+0000","last_fullsized":"2024-04-17T06:10:27.790507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:22:41.057096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792296+0000","last_change":"2024-04-17T06:10:27.786171+0000","last_active":"2024-04-17T06:10:27.792296+0000","last_peered":"2024-04-17T06:10:27.792296+0000","last_clean":"2024-04-17T06:10:27.792296+0000","last_became_active":"2024-04-17T06:10:27.785894+0000","last_became_peered":"2024-04-17T06:10:27.785894+0000","last_unstale":"2024-04-17T06:10:27.792296+0000","last_undegraded":"2024-04-17T06:10:27.792296+0000","last_fullsized":"2024-04-17T06:10:27.792296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.793798+0000","last_change":"2024-04-17T06:10:27.789108+0000","last_active":"2024-04-17T06:10:27.793798+0000","last_peered":"2024-04-17T06:10:27.793798+0000","last_clean":"2024-04-17T06:10:27.793798+0000","last_became_active":"2024-04-17T06:10:27.788829+0000","last_became_peered":"2024-04-17T06:10:27.788829+0000","last_unstale":"2024-04-17T06:10:27.793798+0000","last_undegraded":"2024-04-17T06:10:27.793798+0000","last_fullsized":"2024-04-17T06:10:27.793798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:06.115386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.795958+0000","last_change":"2024-04-17T06:10:27.786698+0000","last_active":"2024-04-17T06:10:27.795958+0000","last_peered":"2024-04-17T06:10:27.795958+0000","last_clean":"2024-04-17T06:10:27.795958+0000","last_became_active":"2024-04-17T06:10:27.786273+0000","last_became_peered":"2024-04-17T06:10:27.786273+0000","last_unstale":"2024-04-17T06:10:27.795958+0000","last_undegraded":"2024-04-17T06:10:27.795958+0000","last_fullsized":"2024-04-17T06:10:27.795958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.532545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.784571+0000","last_change":"2024-04-17T06:10:27.778601+0000","last_active":"2024-04-17T06:10:27.784571+0000","last_peered":"2024-04-17T06:10:27.784571+0000","last_clean":"2024-04-17T06:10:27.784571+0000","last_became_active":"2024-04-17T06:10:27.778335+0000","last_became_peered":"2024-04-17T06:10:27.778335+0000","last_unstale":"2024-04-17T06:10:27.784571+0000","last_undegraded":"2024-04-17T06:10:27.784571+0000","last_fullsized":"2024-04-17T06:10:27.784571+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:13.377063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796278+0000","last_change":"2024-04-17T06:10:27.788159+0000","last_active":"2024-04-17T06:10:27.796278+0000","last_peered":"2024-04-17T06:10:27.796278+0000","last_clean":"2024-04-17T06:10:27.796278+0000","last_became_active":"2024-04-17T06:10:27.787877+0000","last_became_peered":"2024-04-17T06:10:27.787877+0000","last_unstale":"2024-04-17T06:10:27.796278+0000","last_undegraded":"2024-04-17T06:10:27.796278+0000","last_fullsized":"2024-04-17T06:10:27.796278+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:13.072810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797403+0000","last_change":"2024-04-17T06:10:27.788810+0000","last_active":"2024-04-17T06:10:27.797403+0000","last_peered":"2024-04-17T06:10:27.797403+0000","last_clean":"2024-04-17T06:10:27.797403+0000","last_became_active":"2024-04-17T06:10:27.788481+0000","last_became_peered":"2024-04-17T06:10:27.788481+0000","last_unstale":"2024-04-17T06:10:27.797403+0000","last_undegraded":"2024-04-17T06:10:27.797403+0000","last_fullsized":"2024-04-17T06:10:27.797403+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:51.428907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.793141+0000","last_change":"2024-04-17T06:10:27.789579+0000","last_active":"2024-04-17T06:10:27.793141+0000","last_peered":"2024-04-17T06:10:27.793141+0000","last_clean":"2024-04-17T06:10:27.793141+0000","last_became_active":"2024-04-17T06:10:27.789109+0000","last_became_peered":"2024-04-17T06:10:27.789109+0000","last_unstale":"2024-04-17T06:10:27.793141+0000","last_undegraded":"2024-04-17T06:10:27.793141+0000","last_fullsized":"2024-04-17T06:10:27.793141+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.532545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792107+0000","last_change":"2024-04-17T06:10:27.787264+0000","last_active":"2024-04-17T06:10:27.792107+0000","last_peered":"2024-04-17T06:10:27.792107+0000","last_clean":"2024-04-17T06:10:27.792107+0000","last_became_active":"2024-04-17T06:10:27.787008+0000","last_became_peered":"2024-04-17T06:10:27.787008+0000","last_unstale":"2024-04-17T06:10:27.792107+0000","last_undegraded":"2024-04-17T06:10:27.792107+0000","last_fullsized":"2024-04-17T06:10:27.792107+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:32:14.211303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.789487+0000","last_change":"2024-04-17T06:10:27.787860+0000","last_active":"2024-04-17T06:10:27.789487+0000","last_peered":"2024-04-17T06:10:27.789487+0000","last_clean":"2024-04-17T06:10:27.789487+0000","last_became_active":"2024-04-17T06:10:27.787558+0000","last_became_peered":"2024-04-17T06:10:27.787558+0000","last_unstale":"2024-04-17T06:10:27.789487+0000","last_undegraded":"2024-04-17T06:10:27.789487+0000","last_fullsized":"2024-04-17T06:10:27.789487+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:48.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.790292+0000","last_change":"2024-04-17T06:10:27.784417+0000","last_active":"2024-04-17T06:10:27.790292+0000","last_peered":"2024-04-17T06:10:27.790292+0000","last_clean":"2024-04-17T06:10:27.790292+0000","last_became_active":"2024-04-17T06:10:27.784164+0000","last_became_peered":"2024-04-17T06:10:27.784164+0000","last_unstale":"2024-04-17T06:10:27.790292+0000","last_undegraded":"2024-04-17T06:10:27.790292+0000","last_fullsized":"2024-04-17T06:10:27.790292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:58.769842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.786605+0000","last_change":"2024-04-17T06:10:27.783246+0000","last_active":"2024-04-17T06:10:27.786605+0000","last_peered":"2024-04-17T06:10:27.786605+0000","last_clean":"2024-04-17T06:10:27.786605+0000","last_became_active":"2024-04-17T06:10:27.782711+0000","last_became_peered":"2024-04-17T06:10:27.782711+0000","last_unstale":"2024-04-17T06:10:27.786605+0000","last_undegraded":"2024-04-17T06:10:27.786605+0000","last_fullsized":"2024-04-17T06:10:27.786605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:30.540315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798059+0000","last_change":"2024-04-17T06:10:27.792547+0000","last_active":"2024-04-17T06:10:27.798059+0000","last_peered":"2024-04-17T06:10:27.798059+0000","last_clean":"2024-04-17T06:10:27.798059+0000","last_became_active":"2024-04-17T06:10:27.792302+0000","last_became_peered":"2024-04-17T06:10:27.792302+0000","last_unstale":"2024-04-17T06:10:27.798059+0000","last_undegraded":"2024-04-17T06:10:27.798059+0000","last_fullsized":"2024-04-17T06:10:27.798059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:23.566483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796106+0000","last_change":"2024-04-17T06:10:27.790879+0000","last_active":"2024-04-17T06:10:27.796106+0000","last_peered":"2024-04-17T06:10:27.796106+0000","last_clean":"2024-04-17T06:10:27.796106+0000","last_became_active":"2024-04-17T06:10:27.790347+0000","last_became_peered":"2024-04-17T06:10:27.790347+0000","last_unstale":"2024-04-17T06:10:27.796106+0000","last_undegraded":"2024-04-17T06:10:27.796106+0000","last_fullsized":"2024-04-17T06:10:27.796106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:34:11.210087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.784233+0000","last_change":"2024-04-17T06:10:27.777722+0000","last_active":"2024-04-17T06:10:27.784233+0000","last_peered":"2024-04-17T06:10:27.784233+0000","last_clean":"2024-04-17T06:10:27.784233+0000","last_became_active":"2024-04-17T06:10:27.777426+0000","last_became_peered":"2024-04-17T06:10:27.777426+0000","last_unstale":"2024-04-17T06:10:27.784233+0000","last_undegraded":"2024-04-17T06:10:27.784233+0000","last_fullsized":"2024-04-17T06:10:27.784233+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:56.558792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797339+0000","last_change":"2024-04-17T06:10:27.791689+0000","last_active":"2024-04-17T06:10:27.797339+0000","last_peered":"2024-04-17T06:10:27.797339+0000","last_clean":"2024-04-17T06:10:27.797339+0000","last_became_active":"2024-04-17T06:10:27.791266+0000","last_became_peered":"2024-04-17T06:10:27.791266+0000","last_unstale":"2024-04-17T06:10:27.797339+0000","last_undegraded":"2024-04-17T06:10:27.797339+0000","last_fullsized":"2024-04-17T06:10:27.797339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:08.364826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.785348+0000","last_change":"2024-04-17T06:10:27.781355+0000","last_active":"2024-04-17T06:10:27.785348+0000","last_peered":"2024-04-17T06:10:27.785348+0000","last_clean":"2024-04-17T06:10:27.785348+0000","last_became_active":"2024-04-17T06:10:27.779488+0000","last_became_peered":"2024-04-17T06:10:27.779488+0000","last_unstale":"2024-04-17T06:10:27.785348+0000","last_undegraded":"2024-04-17T06:10:27.785348+0000","last_fullsized":"2024-04-17T06:10:27.785348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:59.939671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.802376+0000","last_change":"2024-04-17T06:10:27.797855+0000","last_active":"2024-04-17T06:10:27.802376+0000","last_peered":"2024-04-17T06:10:27.802376+0000","last_clean":"2024-04-17T06:10:27.802376+0000","last_became_active":"2024-04-17T06:10:27.797540+0000","last_became_peered":"2024-04-17T06:10:27.797540+0000","last_unstale":"2024-04-17T06:10:27.802376+0000","last_undegraded":"2024-04-17T06:10:27.802376+0000","last_fullsized":"2024-04-17T06:10:27.802376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:53.568351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.785133+0000","last_change":"2024-04-17T06:10:27.782568+0000","last_active":"2024-04-17T06:10:27.785133+0000","last_peered":"2024-04-17T06:10:27.785133+0000","last_clean":"2024-04-17T06:10:27.785133+0000","last_became_active":"2024-04-17T06:10:27.782308+0000","last_became_peered":"2024-04-17T06:10:27.782308+0000","last_unstale":"2024-04-17T06:10:27.785133+0000","last_undegraded":"2024-04-17T06:10:27.785133+0000","last_fullsized":"2024-04-17T06:10:27.785133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:30.540315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798927+0000","last_change":"2024-04-17T06:10:27.795904+0000","last_active":"2024-04-17T06:10:27.798927+0000","last_peered":"2024-04-17T06:10:27.798927+0000","last_clean":"2024-04-17T06:10:27.798927+0000","last_became_active":"2024-04-17T06:10:27.795629+0000","last_became_peered":"2024-04-17T06:10:27.795629+0000","last_unstale":"2024-04-17T06:10:27.798927+0000","last_undegraded":"2024-04-17T06:10:27.798927+0000","last_fullsized":"2024-04-17T06:10:27.798927+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798079+0000","last_change":"2024-04-17T06:10:27.794083+0000","last_active":"2024-04-17T06:10:27.798079+0000","last_peered":"2024-04-17T06:10:27.798079+0000","last_clean":"2024-04-17T06:10:27.798079+0000","last_became_active":"2024-04-17T06:10:27.793777+0000","last_became_peered":"2024-04-17T06:10:27.793777+0000","last_unstale":"2024-04-17T06:10:27.798079+0000","last_undegraded":"2024-04-17T06:10:27.798079+0000","last_fullsized":"2024-04-17T06:10:27.798079+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:54.076686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797476+0000","last_change":"2024-04-17T06:10:27.791560+0000","last_active":"2024-04-17T06:10:27.797476+0000","last_peered":"2024-04-17T06:10:27.797476+0000","last_clean":"2024-04-17T06:10:27.797476+0000","last_became_active":"2024-04-17T06:10:27.791277+0000","last_became_peered":"2024-04-17T06:10:27.791277+0000","last_unstale":"2024-04-17T06:10:27.797476+0000","last_undegraded":"2024-04-17T06:10:27.797476+0000","last_fullsized":"2024-04-17T06:10:27.797476+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:13.072810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797380+0000","last_change":"2024-04-17T06:10:27.788991+0000","last_active":"2024-04-17T06:10:27.797380+0000","last_peered":"2024-04-17T06:10:27.797380+0000","last_clean":"2024-04-17T06:10:27.797380+0000","last_became_active":"2024-04-17T06:10:27.788251+0000","last_became_peered":"2024-04-17T06:10:27.788251+0000","last_unstale":"2024-04-17T06:10:27.797380+0000","last_undegraded":"2024-04-17T06:10:27.797380+0000","last_fullsized":"2024-04-17T06:10:27.797380+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:38.108803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796679+0000","last_change":"2024-04-17T06:10:26.784120+0000","last_active":"2024-04-17T06:10:27.796679+0000","last_peered":"2024-04-17T06:10:27.796679+0000","last_clean":"2024-04-17T06:10:27.796679+0000","last_became_active":"2024-04-17T06:10:26.783847+0000","last_became_peered":"2024-04-17T06:10:26.783847+0000","last_unstale":"2024-04-17T06:10:27.796679+0000","last_undegraded":"2024-04-17T06:10:27.796679+0000","last_fullsized":"2024-04-17T06:10:27.796679+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:08.112354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797385+0000","last_change":"2024-04-17T06:10:27.791051+0000","last_active":"2024-04-17T06:10:27.797385+0000","last_peered":"2024-04-17T06:10:27.797385+0000","last_clean":"2024-04-17T06:10:27.797385+0000","last_became_active":"2024-04-17T06:10:27.790781+0000","last_became_peered":"2024-04-17T06:10:27.790781+0000","last_unstale":"2024-04-17T06:10:27.797385+0000","last_undegraded":"2024-04-17T06:10:27.797385+0000","last_fullsized":"2024-04-17T06:10:27.797385+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:27.405253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792170+0000","last_change":"2024-04-17T06:10:26.787008+0000","last_active":"2024-04-17T06:10:27.792170+0000","last_peered":"2024-04-17T06:10:27.792170+0000","last_clean":"2024-04-17T06:10:27.792170+0000","last_became_active":"2024-04-17T06:10:26.786742+0000","last_became_peered":"2024-04-17T06:10:26.786742+0000","last_unstale":"2024-04-17T06:10:27.792170+0000","last_undegraded":"2024-04-17T06:10:27.792170+0000","last_fullsized":"2024-04-17T06:10:27.792170+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:58.935115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.795660+0000","last_change":"2024-04-17T06:10:27.787055+0000","last_active":"2024-04-17T06:10:27.795660+0000","last_peered":"2024-04-17T06:10:27.795660+0000","last_clean":"2024-04-17T06:10:27.795660+0000","last_became_active":"2024-04-17T06:10:27.786821+0000","last_became_peered":"2024-04-17T06:10:27.786821+0000","last_unstale":"2024-04-17T06:10:27.795660+0000","last_undegraded":"2024-04-17T06:10:27.795660+0000","last_fullsized":"2024-04-17T06:10:27.795660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796683+0000","last_change":"2024-04-17T06:10:26.777762+0000","last_active":"2024-04-17T06:10:27.796683+0000","last_peered":"2024-04-17T06:10:27.796683+0000","last_clean":"2024-04-17T06:10:27.796683+0000","last_became_active":"2024-04-17T06:10:26.777522+0000","last_became_peered":"2024-04-17T06:10:26.777522+0000","last_unstale":"2024-04-17T06:10:27.796683+0000","last_undegraded":"2024-04-17T06:10:27.796683+0000","last_fullsized":"2024-04-17T06:10:27.796683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:47.393482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797932+0000","last_change":"2024-04-17T06:10:27.793308+0000","last_active":"2024-04-17T06:10:27.797932+0000","last_peered":"2024-04-17T06:10:27.797932+0000","last_clean":"2024-04-17T06:10:27.797932+0000","last_became_active":"2024-04-17T06:10:27.792964+0000","last_became_peered":"2024-04-17T06:10:27.792964+0000","last_unstale":"2024-04-17T06:10:27.797932+0000","last_undegraded":"2024-04-17T06:10:27.797932+0000","last_fullsized":"2024-04-17T06:10:27.797932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:32:06.115386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797359+0000","last_change":"2024-04-17T06:10:26.785844+0000","last_active":"2024-04-17T06:10:27.797359+0000","last_peered":"2024-04-17T06:10:27.797359+0000","last_clean":"2024-04-17T06:10:27.797359+0000","last_became_active":"2024-04-17T06:10:26.785544+0000","last_became_peered":"2024-04-17T06:10:26.785544+0000","last_unstale":"2024-04-17T06:10:27.797359+0000","last_undegraded":"2024-04-17T06:10:27.797359+0000","last_fullsized":"2024-04-17T06:10:27.797359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:58.935115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792428+0000","last_change":"2024-04-17T06:10:27.784936+0000","last_active":"2024-04-17T06:10:27.792428+0000","last_peered":"2024-04-17T06:10:27.792428+0000","last_clean":"2024-04-17T06:10:27.792428+0000","last_became_active":"2024-04-17T06:10:27.784624+0000","last_became_peered":"2024-04-17T06:10:27.784624+0000","last_unstale":"2024-04-17T06:10:27.792428+0000","last_undegraded":"2024-04-17T06:10:27.792428+0000","last_fullsized":"2024-04-17T06:10:27.792428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:48.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797885+0000","last_change":"2024-04-17T06:10:26.786620+0000","last_active":"2024-04-17T06:10:27.797885+0000","last_peered":"2024-04-17T06:10:27.797885+0000","last_clean":"2024-04-17T06:10:27.797885+0000","last_became_active":"2024-04-17T06:10:26.786331+0000","last_became_peered":"2024-04-17T06:10:26.786331+0000","last_unstale":"2024-04-17T06:10:27.797885+0000","last_undegraded":"2024-04-17T06:10:27.797885+0000","last_fullsized":"2024-04-17T06:10:27.797885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:16:37.104248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.786305+0000","last_change":"2024-04-17T06:10:27.782962+0000","last_active":"2024-04-17T06:10:27.786305+0000","last_peered":"2024-04-17T06:10:27.786305+0000","last_clean":"2024-04-17T06:10:27.786305+0000","last_became_active":"2024-04-17T06:10:27.782670+0000","last_became_peered":"2024-04-17T06:10:27.782670+0000","last_unstale":"2024-04-17T06:10:27.786305+0000","last_undegraded":"2024-04-17T06:10:27.786305+0000","last_fullsized":"2024-04-17T06:10:27.786305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:32:24.474707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.799747+0000","last_change":"2024-04-17T06:10:26.782183+0000","last_active":"2024-04-17T06:10:27.799747+0000","last_peered":"2024-04-17T06:10:27.799747+0000","last_clean":"2024-04-17T06:10:27.799747+0000","last_became_active":"2024-04-17T06:10:26.781885+0000","last_became_peered":"2024-04-17T06:10:26.781885+0000","last_unstale":"2024-04-17T06:10:27.799747+0000","last_undegraded":"2024-04-17T06:10:27.799747+0000","last_fullsized":"2024-04-17T06:10:27.799747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:58.935115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.795935+0000","last_change":"2024-04-17T06:10:27.786328+0000","last_active":"2024-04-17T06:10:27.795935+0000","last_peered":"2024-04-17T06:10:27.795935+0000","last_clean":"2024-04-17T06:10:27.795935+0000","last_became_active":"2024-04-17T06:10:27.786049+0000","last_became_peered":"2024-04-17T06:10:27.786049+0000","last_unstale":"2024-04-17T06:10:27.795935+0000","last_undegraded":"2024-04-17T06:10:27.795935+0000","last_fullsized":"2024-04-17T06:10:27.795935+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:48.398037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.781571+0000","last_change":"2024-04-17T06:10:26.782942+0000","last_active":"2024-04-17T06:10:27.781571+0000","last_peered":"2024-04-17T06:10:27.781571+0000","last_clean":"2024-04-17T06:10:27.781571+0000","last_became_active":"2024-04-17T06:10:26.782661+0000","last_became_peered":"2024-04-17T06:10:26.782661+0000","last_unstale":"2024-04-17T06:10:27.781571+0000","last_undegraded":"2024-04-17T06:10:27.781571+0000","last_fullsized":"2024-04-17T06:10:27.781571+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:58.935115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798127+0000","last_change":"2024-04-17T06:10:27.794238+0000","last_active":"2024-04-17T06:10:27.798127+0000","last_peered":"2024-04-17T06:10:27.798127+0000","last_clean":"2024-04-17T06:10:27.798127+0000","last_became_active":"2024-04-17T06:10:27.793714+0000","last_became_peered":"2024-04-17T06:10:27.793714+0000","last_unstale":"2024-04-17T06:10:27.798127+0000","last_undegraded":"2024-04-17T06:10:27.798127+0000","last_fullsized":"2024-04-17T06:10:27.798127+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:37:32.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.775954+0000","last_change":"2024-04-17T06:10:26.780773+0000","last_active":"2024-04-17T06:10:27.775954+0000","last_peered":"2024-04-17T06:10:27.775954+0000","last_clean":"2024-04-17T06:10:27.775954+0000","last_became_active":"2024-04-17T06:10:26.780489+0000","last_became_peered":"2024-04-17T06:10:26.780489+0000","last_unstale":"2024-04-17T06:10:27.775954+0000","last_undegraded":"2024-04-17T06:10:27.775954+0000","last_fullsized":"2024-04-17T06:10:27.775954+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:42:08.112354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.789721+0000","last_change":"2024-04-17T06:10:27.786059+0000","last_active":"2024-04-17T06:10:27.789721+0000","last_peered":"2024-04-17T06:10:27.789721+0000","last_clean":"2024-04-17T06:10:27.789721+0000","last_became_active":"2024-04-17T06:10:27.785780+0000","last_became_peered":"2024-04-17T06:10:27.785780+0000","last_unstale":"2024-04-17T06:10:27.789721+0000","last_undegraded":"2024-04-17T06:10:27.789721+0000","last_fullsized":"2024-04-17T06:10:27.789721+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:05:34.475577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.784030+0000","last_change":"2024-04-17T06:10:26.784585+0000","last_active":"2024-04-17T06:10:27.784030+0000","last_peered":"2024-04-17T06:10:27.784030+0000","last_clean":"2024-04-17T06:10:27.784030+0000","last_became_active":"2024-04-17T06:10:26.784337+0000","last_became_peered":"2024-04-17T06:10:26.784337+0000","last_unstale":"2024-04-17T06:10:27.784030+0000","last_undegraded":"2024-04-17T06:10:27.784030+0000","last_fullsized":"2024-04-17T06:10:27.784030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:20.097285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797855+0000","last_change":"2024-04-17T06:10:27.790494+0000","last_active":"2024-04-17T06:10:27.797855+0000","last_peered":"2024-04-17T06:10:27.797855+0000","last_clean":"2024-04-17T06:10:27.797855+0000","last_became_active":"2024-04-17T06:10:27.790229+0000","last_became_peered":"2024-04-17T06:10:27.790229+0000","last_unstale":"2024-04-17T06:10:27.797855+0000","last_undegraded":"2024-04-17T06:10:27.797855+0000","last_fullsized":"2024-04-17T06:10:27.797855+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:37.229605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.770480+0000","last_change":"2024-04-17T06:10:26.785233+0000","last_active":"2024-04-17T06:10:27.770480+0000","last_peered":"2024-04-17T06:10:27.770480+0000","last_clean":"2024-04-17T06:10:27.770480+0000","last_became_active":"2024-04-17T06:10:26.784966+0000","last_became_peered":"2024-04-17T06:10:26.784966+0000","last_unstale":"2024-04-17T06:10:27.770480+0000","last_undegraded":"2024-04-17T06:10:27.770480+0000","last_fullsized":"2024-04-17T06:10:27.770480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:22.561927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792387+0000","last_change":"2024-04-17T06:10:27.787597+0000","last_active":"2024-04-17T06:10:27.792387+0000","last_peered":"2024-04-17T06:10:27.792387+0000","last_clean":"2024-04-17T06:10:27.792387+0000","last_became_active":"2024-04-17T06:10:27.787247+0000","last_became_peered":"2024-04-17T06:10:27.787247+0000","last_unstale":"2024-04-17T06:10:27.792387+0000","last_undegraded":"2024-04-17T06:10:27.792387+0000","last_fullsized":"2024-04-17T06:10:27.792387+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:08.364826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798975+0000","last_change":"2024-04-17T06:10:26.784372+0000","last_active":"2024-04-17T06:10:27.798975+0000","last_peered":"2024-04-17T06:10:27.798975+0000","last_clean":"2024-04-17T06:10:27.798975+0000","last_became_active":"2024-04-17T06:10:26.784105+0000","last_became_peered":"2024-04-17T06:10:26.784105+0000","last_unstale":"2024-04-17T06:10:27.798975+0000","last_undegraded":"2024-04-17T06:10:27.798975+0000","last_fullsized":"2024-04-17T06:10:27.798975+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:50.424352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.799359+0000","last_change":"2024-04-17T06:10:27.792824+0000","last_active":"2024-04-17T06:10:27.799359+0000","last_peered":"2024-04-17T06:10:27.799359+0000","last_clean":"2024-04-17T06:10:27.799359+0000","last_became_active":"2024-04-17T06:10:27.792546+0000","last_became_peered":"2024-04-17T06:10:27.792546+0000","last_unstale":"2024-04-17T06:10:27.799359+0000","last_undegraded":"2024-04-17T06:10:27.799359+0000","last_fullsized":"2024-04-17T06:10:27.799359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:13.072810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.805165+0000","last_change":"2024-04-17T06:10:26.788487+0000","last_active":"2024-04-17T06:10:27.805165+0000","last_peered":"2024-04-17T06:10:27.805165+0000","last_clean":"2024-04-17T06:10:27.805165+0000","last_became_active":"2024-04-17T06:10:26.787675+0000","last_became_peered":"2024-04-17T06:10:26.787675+0000","last_unstale":"2024-04-17T06:10:27.805165+0000","last_undegraded":"2024-04-17T06:10:27.805165+0000","last_fullsized":"2024-04-17T06:10:27.805165+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:35:57.765286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792447+0000","last_change":"2024-04-17T06:10:27.785528+0000","last_active":"2024-04-17T06:10:27.792447+0000","last_peered":"2024-04-17T06:10:27.792447+0000","last_clean":"2024-04-17T06:10:27.792447+0000","last_became_active":"2024-04-17T06:10:27.785276+0000","last_became_peered":"2024-04-17T06:10:27.785276+0000","last_unstale":"2024-04-17T06:10:27.792447+0000","last_undegraded":"2024-04-17T06:10:27.792447+0000","last_fullsized":"2024-04-17T06:10:27.792447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:16.532545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.793278+0000","last_change":"2024-04-17T06:10:26.788770+0000","last_active":"2024-04-17T06:10:27.793278+0000","last_peered":"2024-04-17T06:10:27.793278+0000","last_clean":"2024-04-17T06:10:27.793278+0000","last_became_active":"2024-04-17T06:10:26.788541+0000","last_became_peered":"2024-04-17T06:10:26.788541+0000","last_unstale":"2024-04-17T06:10:27.793278+0000","last_undegraded":"2024-04-17T06:10:27.793278+0000","last_fullsized":"2024-04-17T06:10:27.793278+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T06:32:23.470152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798140+0000","last_change":"2024-04-17T06:10:27.793976+0000","last_active":"2024-04-17T06:10:27.798140+0000","last_peered":"2024-04-17T06:10:27.798140+0000","last_clean":"2024-04-17T06:10:27.798140+0000","last_became_active":"2024-04-17T06:10:27.793159+0000","last_became_peered":"2024-04-17T06:10:27.793159+0000","last_unstale":"2024-04-17T06:10:27.798140+0000","last_undegraded":"2024-04-17T06:10:27.798140+0000","last_fullsized":"2024-04-17T06:10:27.798140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:53:53.568351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.781848+0000","last_change":"2024-04-17T06:10:26.782356+0000","last_active":"2024-04-17T06:10:27.781848+0000","last_peered":"2024-04-17T06:10:27.781848+0000","last_clean":"2024-04-17T06:10:27.781848+0000","last_became_active":"2024-04-17T06:10:26.782011+0000","last_became_peered":"2024-04-17T06:10:26.782011+0000","last_unstale":"2024-04-17T06:10:27.781848+0000","last_undegraded":"2024-04-17T06:10:27.781848+0000","last_fullsized":"2024-04-17T06:10:27.781848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:15.527989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797375+0000","last_change":"2024-04-17T06:10:27.791932+0000","last_active":"2024-04-17T06:10:27.797375+0000","last_peered":"2024-04-17T06:10:27.797375+0000","last_clean":"2024-04-17T06:10:27.797375+0000","last_became_active":"2024-04-17T06:10:27.791662+0000","last_became_peered":"2024-04-17T06:10:27.791662+0000","last_unstale":"2024-04-17T06:10:27.797375+0000","last_undegraded":"2024-04-17T06:10:27.797375+0000","last_fullsized":"2024-04-17T06:10:27.797375+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:08.364826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.769813+0000","last_change":"2024-04-17T06:10:26.784025+0000","last_active":"2024-04-17T06:10:27.769813+0000","last_peered":"2024-04-17T06:10:27.769813+0000","last_clean":"2024-04-17T06:10:27.769813+0000","last_became_active":"2024-04-17T06:10:26.783742+0000","last_became_peered":"2024-04-17T06:10:26.783742+0000","last_unstale":"2024-04-17T06:10:27.769813+0000","last_undegraded":"2024-04-17T06:10:27.769813+0000","last_fullsized":"2024-04-17T06:10:27.769813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:51:40.914371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798106+0000","last_change":"2024-04-17T06:10:27.794031+0000","last_active":"2024-04-17T06:10:27.798106+0000","last_peered":"2024-04-17T06:10:27.798106+0000","last_clean":"2024-04-17T06:10:27.798106+0000","last_became_active":"2024-04-17T06:10:27.793683+0000","last_became_peered":"2024-04-17T06:10:27.793683+0000","last_unstale":"2024-04-17T06:10:27.798106+0000","last_undegraded":"2024-04-17T06:10:27.798106+0000","last_fullsized":"2024-04-17T06:10:27.798106+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:13.408047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.803406+0000","last_change":"2024-04-17T06:10:26.781122+0000","last_active":"2024-04-17T06:10:27.803406+0000","last_peered":"2024-04-17T06:10:27.803406+0000","last_clean":"2024-04-17T06:10:27.803406+0000","last_became_active":"2024-04-17T06:10:26.780824+0000","last_became_peered":"2024-04-17T06:10:26.780824+0000","last_unstale":"2024-04-17T06:10:27.803406+0000","last_undegraded":"2024-04-17T06:10:27.803406+0000","last_fullsized":"2024-04-17T06:10:27.803406+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:03:15.527989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797765+0000","last_change":"2024-04-17T06:10:27.793828+0000","last_active":"2024-04-17T06:10:27.797765+0000","last_peered":"2024-04-17T06:10:27.797765+0000","last_clean":"2024-04-17T06:10:27.797765+0000","last_became_active":"2024-04-17T06:10:27.793501+0000","last_became_peered":"2024-04-17T06:10:27.793501+0000","last_unstale":"2024-04-17T06:10:27.797765+0000","last_undegraded":"2024-04-17T06:10:27.797765+0000","last_fullsized":"2024-04-17T06:10:27.797765+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:13:48.399109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.802740+0000","last_change":"2024-04-17T06:10:26.782705+0000","last_active":"2024-04-17T06:10:27.802740+0000","last_peered":"2024-04-17T06:10:27.802740+0000","last_clean":"2024-04-17T06:10:27.802740+0000","last_became_active":"2024-04-17T06:10:26.781684+0000","last_became_peered":"2024-04-17T06:10:26.781684+0000","last_unstale":"2024-04-17T06:10:27.802740+0000","last_undegraded":"2024-04-17T06:10:27.802740+0000","last_fullsized":"2024-04-17T06:10:27.802740+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:08:53.072130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.786011+0000","last_change":"2024-04-17T06:10:27.782130+0000","last_active":"2024-04-17T06:10:27.786011+0000","last_peered":"2024-04-17T06:10:27.786011+0000","last_clean":"2024-04-17T06:10:27.786011+0000","last_became_active":"2024-04-17T06:10:27.781825+0000","last_became_peered":"2024-04-17T06:10:27.781825+0000","last_unstale":"2024-04-17T06:10:27.786011+0000","last_undegraded":"2024-04-17T06:10:27.786011+0000","last_fullsized":"2024-04-17T06:10:27.786011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:12:21.101840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792343+0000","last_change":"2024-04-17T06:10:26.779973+0000","last_active":"2024-04-17T06:10:27.792343+0000","last_peered":"2024-04-17T06:10:27.792343+0000","last_clean":"2024-04-17T06:10:27.792343+0000","last_became_active":"2024-04-17T06:10:26.779656+0000","last_became_peered":"2024-04-17T06:10:26.779656+0000","last_unstale":"2024-04-17T06:10:27.792343+0000","last_undegraded":"2024-04-17T06:10:27.792343+0000","last_fullsized":"2024-04-17T06:10:27.792343+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:31.670808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.795759+0000","last_change":"2024-04-17T06:10:27.790149+0000","last_active":"2024-04-17T06:10:27.795759+0000","last_peered":"2024-04-17T06:10:27.795759+0000","last_clean":"2024-04-17T06:10:27.795759+0000","last_became_active":"2024-04-17T06:10:27.787463+0000","last_became_peered":"2024-04-17T06:10:27.787463+0000","last_unstale":"2024-04-17T06:10:27.795759+0000","last_undegraded":"2024-04-17T06:10:27.795759+0000","last_fullsized":"2024-04-17T06:10:27.795759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:27.405253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.789898+0000","last_change":"2024-04-17T06:10:26.779273+0000","last_active":"2024-04-17T06:10:27.789898+0000","last_peered":"2024-04-17T06:10:27.789898+0000","last_clean":"2024-04-17T06:10:27.789898+0000","last_became_active":"2024-04-17T06:10:26.779026+0000","last_became_peered":"2024-04-17T06:10:26.779026+0000","last_unstale":"2024-04-17T06:10:27.789898+0000","last_undegraded":"2024-04-17T06:10:27.789898+0000","last_fullsized":"2024-04-17T06:10:27.789898+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:06:33.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798398+0000","last_change":"2024-04-17T06:10:27.794013+0000","last_active":"2024-04-17T06:10:27.798398+0000","last_peered":"2024-04-17T06:10:27.798398+0000","last_clean":"2024-04-17T06:10:27.798398+0000","last_became_active":"2024-04-17T06:10:27.793756+0000","last_became_peered":"2024-04-17T06:10:27.793756+0000","last_unstale":"2024-04-17T06:10:27.798398+0000","last_undegraded":"2024-04-17T06:10:27.798398+0000","last_fullsized":"2024-04-17T06:10:27.798398+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:58:30.129352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.781597+0000","last_change":"2024-04-17T06:10:26.778012+0000","last_active":"2024-04-17T06:10:27.781597+0000","last_peered":"2024-04-17T06:10:27.781597+0000","last_clean":"2024-04-17T06:10:27.781597+0000","last_became_active":"2024-04-17T06:10:26.777094+0000","last_became_peered":"2024-04-17T06:10:26.777094+0000","last_unstale":"2024-04-17T06:10:27.781597+0000","last_undegraded":"2024-04-17T06:10:27.781597+0000","last_fullsized":"2024-04-17T06:10:27.781597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:29.535760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798117+0000","last_change":"2024-04-17T06:10:27.794492+0000","last_active":"2024-04-17T06:10:27.798117+0000","last_peered":"2024-04-17T06:10:27.798117+0000","last_clean":"2024-04-17T06:10:27.798117+0000","last_became_active":"2024-04-17T06:10:27.794290+0000","last_became_peered":"2024-04-17T06:10:27.794290+0000","last_unstale":"2024-04-17T06:10:27.798117+0000","last_undegraded":"2024-04-17T06:10:27.798117+0000","last_fullsized":"2024-04-17T06:10:27.798117+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:11.621227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.790175+0000","last_change":"2024-04-17T06:10:26.787665+0000","last_active":"2024-04-17T06:10:27.790175+0000","last_peered":"2024-04-17T06:10:27.790175+0000","last_clean":"2024-04-17T06:10:27.790175+0000","last_became_active":"2024-04-17T06:10:26.787080+0000","last_became_peered":"2024-04-17T06:10:26.787080+0000","last_unstale":"2024-04-17T06:10:27.790175+0000","last_undegraded":"2024-04-17T06:10:27.790175+0000","last_fullsized":"2024-04-17T06:10:27.790175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:20:45.709078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792662+0000","last_change":"2024-04-17T06:10:27.787597+0000","last_active":"2024-04-17T06:10:27.792662+0000","last_peered":"2024-04-17T06:10:27.792662+0000","last_clean":"2024-04-17T06:10:27.792662+0000","last_became_active":"2024-04-17T06:10:27.787255+0000","last_became_peered":"2024-04-17T06:10:27.787255+0000","last_unstale":"2024-04-17T06:10:27.792662+0000","last_undegraded":"2024-04-17T06:10:27.792662+0000","last_fullsized":"2024-04-17T06:10:27.792662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:46:13.072810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.771128+0000","last_change":"2024-04-17T06:10:26.782206+0000","last_active":"2024-04-17T06:10:27.771128+0000","last_peered":"2024-04-17T06:10:27.771128+0000","last_clean":"2024-04-17T06:10:27.771128+0000","last_became_active":"2024-04-17T06:10:26.781863+0000","last_became_peered":"2024-04-17T06:10:26.781863+0000","last_unstale":"2024-04-17T06:10:27.771128+0000","last_undegraded":"2024-04-17T06:10:27.771128+0000","last_fullsized":"2024-04-17T06:10:27.771128+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:43:11.146601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.776349+0000","last_change":"2024-04-17T06:10:27.774508+0000","last_active":"2024-04-17T06:10:27.776349+0000","last_peered":"2024-04-17T06:10:27.776349+0000","last_clean":"2024-04-17T06:10:27.776349+0000","last_became_active":"2024-04-17T06:10:27.774209+0000","last_became_peered":"2024-04-17T06:10:27.774209+0000","last_unstale":"2024-04-17T06:10:27.776349+0000","last_undegraded":"2024-04-17T06:10:27.776349+0000","last_fullsized":"2024-04-17T06:10:27.776349+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T08:40:37.229605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.777174+0000","last_change":"2024-04-17T06:10:26.780639+0000","last_active":"2024-04-17T06:10:27.777174+0000","last_peered":"2024-04-17T06:10:27.777174+0000","last_clean":"2024-04-17T06:10:27.777174+0000","last_became_active":"2024-04-17T06:10:26.780362+0000","last_became_peered":"2024-04-17T06:10:26.780362+0000","last_unstale":"2024-04-17T06:10:27.777174+0000","last_undegraded":"2024-04-17T06:10:27.777174+0000","last_fullsized":"2024-04-17T06:10:27.777174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:04:47.393482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798145+0000","last_change":"2024-04-17T06:10:27.794238+0000","last_active":"2024-04-17T06:10:27.798145+0000","last_peered":"2024-04-17T06:10:27.798145+0000","last_clean":"2024-04-17T06:10:27.798145+0000","last_became_active":"2024-04-17T06:10:27.793955+0000","last_became_peered":"2024-04-17T06:10:27.793955+0000","last_unstale":"2024-04-17T06:10:27.798145+0000","last_undegraded":"2024-04-17T06:10:27.798145+0000","last_fullsized":"2024-04-17T06:10:27.798145+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T12:41:56.939325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.770970+0000","last_change":"2024-04-17T06:10:26.786485+0000","last_active":"2024-04-17T06:10:27.770970+0000","last_peered":"2024-04-17T06:10:27.770970+0000","last_clean":"2024-04-17T06:10:27.770970+0000","last_became_active":"2024-04-17T06:10:26.786185+0000","last_became_peered":"2024-04-17T06:10:26.786185+0000","last_unstale":"2024-04-17T06:10:27.770970+0000","last_undegraded":"2024-04-17T06:10:27.770970+0000","last_fullsized":"2024-04-17T06:10:27.770970+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T16:02:12.998924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797360+0000","last_change":"2024-04-17T06:10:27.791808+0000","last_active":"2024-04-17T06:10:27.797360+0000","last_peered":"2024-04-17T06:10:27.797360+0000","last_clean":"2024-04-17T06:10:27.797360+0000","last_became_active":"2024-04-17T06:10:27.791518+0000","last_became_peered":"2024-04-17T06:10:27.791518+0000","last_unstale":"2024-04-17T06:10:27.797360+0000","last_undegraded":"2024-04-17T06:10:27.797360+0000","last_fullsized":"2024-04-17T06:10:27.797360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:42:23.566483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.799478+0000","last_change":"2024-04-17T06:10:26.785898+0000","last_active":"2024-04-17T06:10:27.799478+0000","last_peered":"2024-04-17T06:10:27.799478+0000","last_clean":"2024-04-17T06:10:27.799478+0000","last_became_active":"2024-04-17T06:10:26.785529+0000","last_became_peered":"2024-04-17T06:10:26.785529+0000","last_unstale":"2024-04-17T06:10:27.799478+0000","last_undegraded":"2024-04-17T06:10:27.799478+0000","last_fullsized":"2024-04-17T06:10:27.799478+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:55:10.616672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.797240+0000","last_change":"2024-04-17T06:10:27.787649+0000","last_active":"2024-04-17T06:10:27.797240+0000","last_peered":"2024-04-17T06:10:27.797240+0000","last_clean":"2024-04-17T06:10:27.797240+0000","last_became_active":"2024-04-17T06:10:27.787354+0000","last_became_peered":"2024-04-17T06:10:27.787354+0000","last_unstale":"2024-04-17T06:10:27.797240+0000","last_undegraded":"2024-04-17T06:10:27.797240+0000","last_fullsized":"2024-04-17T06:10:27.797240+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T11:37:27.405253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.771400+0000","last_change":"2024-04-17T06:10:26.781476+0000","last_active":"2024-04-17T06:10:27.771400+0000","last_peered":"2024-04-17T06:10:27.771400+0000","last_clean":"2024-04-17T06:10:27.771400+0000","last_became_active":"2024-04-17T06:10:26.781228+0000","last_became_peered":"2024-04-17T06:10:26.781228+0000","last_unstale":"2024-04-17T06:10:27.771400+0000","last_undegraded":"2024-04-17T06:10:27.771400+0000","last_fullsized":"2024-04-17T06:10:27.771400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:30:46.790174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.800215+0000","last_change":"2024-04-17T06:10:27.797688+0000","last_active":"2024-04-17T06:10:27.800215+0000","last_peered":"2024-04-17T06:10:27.800215+0000","last_clean":"2024-04-17T06:10:27.800215+0000","last_became_active":"2024-04-17T06:10:27.797379+0000","last_became_peered":"2024-04-17T06:10:27.797379+0000","last_unstale":"2024-04-17T06:10:27.800215+0000","last_undegraded":"2024-04-17T06:10:27.800215+0000","last_fullsized":"2024-04-17T06:10:27.800215+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T13:13:28.568615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.776078+0000","last_change":"2024-04-17T06:10:26.791062+0000","last_active":"2024-04-17T06:10:27.776078+0000","last_peered":"2024-04-17T06:10:27.776078+0000","last_clean":"2024-04-17T06:10:27.776078+0000","last_became_active":"2024-04-17T06:10:26.790745+0000","last_became_peered":"2024-04-17T06:10:26.790745+0000","last_unstale":"2024-04-17T06:10:27.776078+0000","last_undegraded":"2024-04-17T06:10:27.776078+0000","last_fullsized":"2024-04-17T06:10:27.776078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:29.535760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.798073+0000","last_change":"2024-04-17T06:10:27.793357+0000","last_active":"2024-04-17T06:10:27.798073+0000","last_peered":"2024-04-17T06:10:27.798073+0000","last_clean":"2024-04-17T06:10:27.798073+0000","last_became_active":"2024-04-17T06:10:27.792995+0000","last_became_peered":"2024-04-17T06:10:27.792995+0000","last_unstale":"2024-04-17T06:10:27.798073+0000","last_undegraded":"2024-04-17T06:10:27.798073+0000","last_fullsized":"2024-04-17T06:10:27.798073+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T09:36:51.428907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.774589+0000","last_change":"2024-04-17T06:10:26.786334+0000","last_active":"2024-04-17T06:10:27.774589+0000","last_peered":"2024-04-17T06:10:27.774589+0000","last_clean":"2024-04-17T06:10:27.774589+0000","last_became_active":"2024-04-17T06:10:26.785955+0000","last_became_peered":"2024-04-17T06:10:26.785955+0000","last_unstale":"2024-04-17T06:10:27.774589+0000","last_undegraded":"2024-04-17T06:10:27.774589+0000","last_fullsized":"2024-04-17T06:10:27.774589+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:55.554236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.792509+0000","last_change":"2024-04-17T06:10:27.787841+0000","last_active":"2024-04-17T06:10:27.792509+0000","last_peered":"2024-04-17T06:10:27.792509+0000","last_clean":"2024-04-17T06:10:27.792509+0000","last_became_active":"2024-04-17T06:10:27.787587+0000","last_became_peered":"2024-04-17T06:10:27.787587+0000","last_unstale":"2024-04-17T06:10:27.792509+0000","last_undegraded":"2024-04-17T06:10:27.792509+0000","last_fullsized":"2024-04-17T06:10:27.792509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.771092+0000","last_change":"2024-04-17T06:10:26.786793+0000","last_active":"2024-04-17T06:10:27.771092+0000","last_peered":"2024-04-17T06:10:27.771092+0000","last_clean":"2024-04-17T06:10:27.771092+0000","last_became_active":"2024-04-17T06:10:26.786124+0000","last_became_peered":"2024-04-17T06:10:26.786124+0000","last_unstale":"2024-04-17T06:10:27.771092+0000","last_undegraded":"2024-04-17T06:10:27.771092+0000","last_fullsized":"2024-04-17T06:10:27.771092+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:08:31.670808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.802501+0000","last_change":"2024-04-17T06:10:27.797912+0000","last_active":"2024-04-17T06:10:27.802501+0000","last_peered":"2024-04-17T06:10:27.802501+0000","last_clean":"2024-04-17T06:10:27.802501+0000","last_became_active":"2024-04-17T06:10:27.797641+0000","last_became_peered":"2024-04-17T06:10:27.797641+0000","last_unstale":"2024-04-17T06:10:27.802501+0000","last_undegraded":"2024-04-17T06:10:27.802501+0000","last_fullsized":"2024-04-17T06:10:27.802501+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:43:08.364826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.777640+0000","last_change":"2024-04-17T06:10:26.785181+0000","last_active":"2024-04-17T06:10:27.777640+0000","last_peered":"2024-04-17T06:10:27.777640+0000","last_clean":"2024-04-17T06:10:27.777640+0000","last_became_active":"2024-04-17T06:10:26.784736+0000","last_became_peered":"2024-04-17T06:10:26.784736+0000","last_unstale":"2024-04-17T06:10:27.777640+0000","last_undegraded":"2024-04-17T06:10:27.777640+0000","last_fullsized":"2024-04-17T06:10:27.777640+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T10:50:55.554236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.796702+0000","last_change":"2024-04-17T06:10:27.786963+0000","last_active":"2024-04-17T06:10:27.796702+0000","last_peered":"2024-04-17T06:10:27.796702+0000","last_clean":"2024-04-17T06:10:27.796702+0000","last_became_active":"2024-04-17T06:10:27.786671+0000","last_became_peered":"2024-04-17T06:10:27.786671+0000","last_unstale":"2024-04-17T06:10:27.796702+0000","last_undegraded":"2024-04-17T06:10:27.796702+0000","last_fullsized":"2024-04-17T06:10:27.796702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T15:32:06.305519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.783621+0000","last_change":"2024-04-17T06:10:26.783829+0000","last_active":"2024-04-17T06:10:27.783621+0000","last_peered":"2024-04-17T06:10:27.783621+0000","last_clean":"2024-04-17T06:10:27.783621+0000","last_became_active":"2024-04-17T06:10:26.783581+0000","last_became_peered":"2024-04-17T06:10:26.783581+0000","last_unstale":"2024-04-17T06:10:27.783621+0000","last_undegraded":"2024-04-17T06:10:27.783621+0000","last_fullsized":"2024-04-17T06:10:27.783621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T18:03:12.372508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.794021+0000","last_change":"2024-04-17T06:10:27.791774+0000","last_active":"2024-04-17T06:10:27.794021+0000","last_peered":"2024-04-17T06:10:27.794021+0000","last_clean":"2024-04-17T06:10:27.794021+0000","last_became_active":"2024-04-17T06:10:27.791474+0000","last_became_peered":"2024-04-17T06:10:27.791474+0000","last_unstale":"2024-04-17T06:10:27.794021+0000","last_undegraded":"2024-04-17T06:10:27.794021+0000","last_fullsized":"2024-04-17T06:10:27.794021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:26.758358+0000","last_clean_scrub_stamp":"2024-04-17T06:10:26.758358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T07:17:48.774295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.776067+0000","last_change":"2024-04-17T06:10:26.786068+0000","last_active":"2024-04-17T06:10:27.776067+0000","last_peered":"2024-04-17T06:10:27.776067+0000","last_clean":"2024-04-17T06:10:27.776067+0000","last_became_active":"2024-04-17T06:10:26.785723+0000","last_became_peered":"2024-04-17T06:10:26.785723+0000","last_unstale":"2024-04-17T06:10:27.776067+0000","last_undegraded":"2024-04-17T06:10:27.776067+0000","last_fullsized":"2024-04-17T06:10:27.776067+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:10:25.753802+0000","last_clean_scrub_stamp":"2024-04-17T06:10:25.753802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T17:41:29.535760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":200,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-17T06:10:27.777762+0000","last_change":"2024-04-17T06:09:09.067035+0000","last_active":"2024-04-17T06:10:27.777762+0000","last_peered":"2024-04-17T06:10:27.777762+0000","last_clean":"2024-04-17T06:10:27.777762+0000","last_became_active":"2024-04-17T06:09:09.066130+0000","last_became_peered":"2024-04-17T06:09:09.066130+0000","last_unstale":"2024-04-17T06:10:27.777762+0000","last_undegraded":"2024-04-17T06:10:27.777762+0000","last_fullsized":"2024-04-17T06:10:27.777762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-17T06:09:07.958875+0000","last_clean_scrub_stamp":"2024-04-17T06:09:07.958875+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T14:32:03.277836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":430080,"data_stored":183398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-17T06:10:31.116 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-17T06:10:31.116 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-17T06:10:31.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.264+0000 7f676337c640 1 Processor -- start 2024-04-17T06:10:31.262 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.264+0000 7f676337c640 1 -- start start 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f676337c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c0b6200 0x7f675c06fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f676337c640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f675c0ccbb0 con 0x7f675c0cbcc0 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f676337c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f675c0cc310 con 0x7f675c15b890 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f676337c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f675c15f3a0 con 0x7f675c0b6200 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6760917640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f675c0cbcc0 0x7f675c0b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:41890/0 (socket says 172.21.15.135:41890) 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6760917640 1 -- 172.21.15.135:0/1554543987 learned_addr learned my addr 172.21.15.135:0/1554543987 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6761919640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c0b6200 0x7f675c06fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2798800187 0 0) 0x7f675c0ccbb0 con 0x7f675c0cbcc0 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6748003520 con 0x7f675c0cbcc0 2024-04-17T06:10:31.266 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258437117 0 0) 0x7f675c0cc310 con 0x7f675c15b890 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f675c0ccbb0 con 0x7f675c15b890 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 936288532 0 0) 0x7f6748003520 con 0x7f675c0cbcc0 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c0b6200 msgr2=0x7f675c06fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c0b6200 0x7f675c06fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 >> v1:172.21.15.190:6789/0 conn(0x7f675c15b890 legacy=0x7f675c071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c0cc5f0 con 0x7f675c0cbcc0 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.265+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f674c001670 con 0x7f675c0cbcc0 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 >> v1:172.21.15.135:6789/0 conn(0x7f675c0cbcc0 legacy=0x7f675c0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6753fff640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6748003b70 0x7f6748003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6753fff640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f6748004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6753fff640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c0b6200 0x7f67480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f675c15f3a0 con 0x7f6748004800 2024-04-17T06:10:31.267 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f674800ba70 con 0x7f6748003b70 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f674800bc70 con 0x7f675c0b6200 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6760917640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f6748004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6761118640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6748003b70 0x7f6748003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6761919640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c0b6200 0x7f67480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6760917640 1 -- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c0b6200 msgr2=0x7f67480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6760917640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c0b6200 0x7f67480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6760917640 1 -- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6748003b70 msgr2=0x7f6748003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6760917640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6748003b70 0x7f6748003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.266+0000 7f6760917640 1 -- 172.21.15.135:0/1554543987 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6748003520 con 0x7f6748004800 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f6760917640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f6748004c80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f674c007c80 tx=0x7f6748005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6c1476b1b99bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c007cc0 con 0x7f6748004800 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674c032070 con 0x7f6748004800 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f6753fff640 1 -- 172.21.15.135:0/1554543987 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0058d0 con 0x7f6748004800 2024-04-17T06:10:31.268 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f676337c640 1 -- 172.21.15.135:0/1554543987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 msgr2=0x7f6748004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f676337c640 1 --2- 172.21.15.135:0/1554543987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f6748004c80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f674c007c80 tx=0x7f6748005830 comp rx=0 tx=0).stop 2024-04-17T06:10:31.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f6761919640 1 -- 172.21.15.135:0/1554543987 reap_dead start 2024-04-17T06:10:31.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f676337c640 1 -- 172.21.15.135:0/1554543987 shutdown_connections 2024-04-17T06:10:31.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.267+0000 7f676337c640 1 -- 172.21.15.135:0/1554543987 >> 172.21.15.135:0/1554543987 conn(0x7f675c0cced0 msgr2=0x7f675c0cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:31.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.268+0000 7f676337c640 1 -- 172.21.15.135:0/1554543987 shutdown_connections 2024-04-17T06:10:31.269 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.268+0000 7f676337c640 1 -- 172.21.15.135:0/1554543987 wait complete. 2024-04-17T06:10:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.269+0000 7f676337c640 1 Processor -- start 2024-04-17T06:10:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.269+0000 7f676337c640 1 -- start start 2024-04-17T06:10:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.269+0000 7f676337c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f675c1f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.269+0000 7f676337c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c1484b0 0x7f675c1450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.269+0000 7f676337c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0cf050 0x7f675c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.269+0000 7f676337c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f675c06fd30 con 0x7f6748004800 2024-04-17T06:10:31.272 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.269+0000 7f676337c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f675c0725b0 con 0x7f675c0cf050 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.269+0000 7f676337c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f675c06feb0 con 0x7f675c1484b0 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f675c1f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f675c1f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54816/0 (socket says 172.21.15.135:54816) 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 -- 172.21.15.135:0/3548235487 learned_addr learned my addr 172.21.15.135:0/3548235487 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761919640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0cf050 0x7f675c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6760917640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c1484b0 0x7f675c1450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 -- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c1484b0 msgr2=0x7f675c1450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c1484b0 0x7f675c1450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 -- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0cf050 msgr2=0x7f675c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0cf050 0x7f675c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 -- 172.21.15.135:0/3548235487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c1f48d0 con 0x7f6748004800 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6760917640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c1484b0 0x7f675c1450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6761118640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f675c1f3970 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f675800f7d0 tx=0x7f675800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:31.273 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6751ffb640 1 -- 172.21.15.135:0/3548235487 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675801a070 con 0x7f6748004800 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6751ffb640 1 -- 172.21.15.135:0/3548235487 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675800fe80 con 0x7f6748004800 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c148930 con 0x7f6748004800 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f6751ffb640 1 -- 172.21.15.135:0/3548235487 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758007050 con 0x7f6748004800 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.270+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1f4310 con 0x7f6748004800 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.271+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675c074940 con 0x7f6748004800 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.272+0000 7f6751ffb640 1 -- 172.21.15.135:0/3548235487 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f675801d070 con 0x7f6748004800 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.272+0000 7f6751ffb640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f674006c920 0x7f674006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.272+0000 7f6751ffb640 1 -- 172.21.15.135:0/3548235487 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f675808b470 con 0x7f6748004800 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.272+0000 7f6760917640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f674006c920 0x7f674006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.274 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.273+0000 7f6760917640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f674006c920 0x7f674006ede0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f675c0d00b0 tx=0x7f674c03b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:31.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.277+0000 7f6751ffb640 1 -- 172.21.15.135:0/3548235487 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6758059710 con 0x7f6748004800 2024-04-17T06:10:31.514 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.516+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f675c072fd0 con 0x7f6748004800 2024-04-17T06:10:31.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.518+0000 7f6751ffb640 1 -- 172.21.15.135:0/3548235487 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f6758059010 con 0x7f6748004800 2024-04-17T06:10:31.516 INFO:teuthology.orchestra.run.smithi135.stdout:[client.0] 2024-04-17T06:10:31.517 INFO:teuthology.orchestra.run.smithi135.stdout: key = AQBWZx9mgVoAEBAAlX2QvDOCLGvYpV4AcvC0GQ== 2024-04-17T06:10:31.517 INFO:teuthology.orchestra.run.smithi135.stdout: caps mds = "allow" 2024-04-17T06:10:31.517 INFO:teuthology.orchestra.run.smithi135.stdout: caps mon = "allow r" 2024-04-17T06:10:31.517 INFO:teuthology.orchestra.run.smithi135.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-17T06:10:31.517 INFO:teuthology.orchestra.run.smithi135.stderr:updated caps for client.0 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f674006c920 msgr2=0x7f674006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f674006c920 0x7f674006ede0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f675c0d00b0 tx=0x7f674c03b040 comp rx=0 tx=0).stop 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 msgr2=0x7f675c1f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f675c1f3970 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f675800f7d0 tx=0x7f675800fc90 comp rx=0 tx=0).stop 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 shutdown_connections 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f675c0cf050 0x7f675c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f675c1484b0 0x7f675c1450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f674006c920 0x7f674006ede0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 --2- 172.21.15.135:0/3548235487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6748004800 0x7f675c1f3970 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.520+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 >> 172.21.15.135:0/3548235487 conn(0x7f675c0cced0 msgr2=0x7f675c15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:31.519 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.521+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 shutdown_connections 2024-04-17T06:10:31.520 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.521+0000 7f676337c640 1 -- 172.21.15.135:0/3548235487 wait complete. 2024-04-17T06:10:31.530 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-17T06:10:31.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.634+0000 7fd4b03ca640 1 Processor -- start 2024-04-17T06:10:31.633 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.635+0000 7fd4b03ca640 1 -- start start 2024-04-17T06:10:31.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.635+0000 7fd4b03ca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a809b100 0x7fd4a80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.635+0000 7fd4b03ca640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd4a80cd2b0 0x7fd4a80ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.635+0000 7fd4b03ca640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a80cd790 con 0x7fd4a81601f0 2024-04-17T06:10:31.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.635+0000 7fd4b03ca640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a80cda70 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.635+0000 7fd4b03ca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a80cdd50 con 0x7fd4a809b100 2024-04-17T06:10:31.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.636+0000 7fd4ae967640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd4a80cd2b0 0x7fd4a80ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.634 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.636+0000 7fd4ae967640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fd4a80cd2b0 0x7fd4a80ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54822/0 (socket says 172.21.15.135:54822) 2024-04-17T06:10:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.636+0000 7fd4ae967640 1 -- 172.21.15.135:0/2113420701 learned_addr learned my addr 172.21.15.135:0/2113420701 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.636+0000 7fd4ad965640 1 --2- 172.21.15.135:0/2113420701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a809b100 0x7fd4a80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.636+0000 7fd4ae967640 1 -- 172.21.15.135:0/2113420701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a809b100 msgr2=0x7fd4a80d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.636+0000 7fd4ae967640 1 --2- 172.21.15.135:0/2113420701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a809b100 0x7fd4a80d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.636+0000 7fd4ae967640 1 -- 172.21.15.135:0/2113420701 >> v1:172.21.15.190:6789/0 conn(0x7fd4a81601f0 legacy=0x7fd4a809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.635 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.636+0000 7fd4ae967640 1 -- 172.21.15.135:0/2113420701 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a80ce030 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4ae967640 1 --2- 172.21.15.135:0/2113420701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a80ce830 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009e50 tx=0x7fd4a002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25068a688cb824fc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:31.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4ad164640 1 -- 172.21.15.135:0/2113420701 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a002cb90 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.636 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4ad164640 1 -- 172.21.15.135:0/2113420701 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a002cd30 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4ad164640 1 -- 172.21.15.135:0/2113420701 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0034610 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4b03ca640 1 -- 172.21.15.135:0/2113420701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 msgr2=0x7fd4a80ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/2113420701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a80ce830 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009e50 tx=0x7fd4a002bff0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4b03ca640 1 -- 172.21.15.135:0/2113420701 shutdown_connections 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/2113420701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a80ce830 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/2113420701 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a809b100 0x7fd4a80d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.637+0000 7fd4b03ca640 1 -- 172.21.15.135:0/2113420701 >> 172.21.15.135:0/2113420701 conn(0x7fd4a80b4190 msgr2=0x7fd4a80b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.638+0000 7fd4b03ca640 1 -- 172.21.15.135:0/2113420701 shutdown_connections 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.638+0000 7fd4b03ca640 1 -- 172.21.15.135:0/2113420701 wait complete. 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4b03ca640 1 Processor -- start 2024-04-17T06:10:31.637 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4b03ca640 1 -- start start 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4b03ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4a809b100 0x7fd4a80cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4b03ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a8155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4b03ca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a81601f0 0x7fd4a80ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4b03ca640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a80d0480 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4b03ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a80d0300 con 0x7fd4a809b100 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4b03ca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a80b1aa0 con 0x7fd4a81601f0 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4ad965640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a8155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4ad965640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a8155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54834/0 (socket says 172.21.15.135:54834) 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.639+0000 7fd4ad965640 1 -- 172.21.15.135:0/384670265 learned_addr learned my addr 172.21.15.135:0/384670265 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ae967640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a81601f0 0x7fd4a80ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.640 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ae166640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4a809b100 0x7fd4a80cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ad965640 1 -- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a81601f0 msgr2=0x7fd4a80ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ad965640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a81601f0 0x7fd4a80ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ad965640 1 -- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4a809b100 msgr2=0x7fd4a80cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ad965640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4a809b100 0x7fd4a80cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ad965640 1 -- 172.21.15.135:0/384670265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a80ca6c0 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ae166640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4a809b100 0x7fd4a80cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4ad965640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a8155c90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400d800 tx=0x7fd4a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd49affd640 1 -- 172.21.15.135:0/384670265 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400b6c0 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd49affd640 1 -- 172.21.15.135:0/384670265 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a4019070 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a80cb830 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd49affd640 1 -- 172.21.15.135:0/384670265 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400bac0 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.640+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a8156810 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.641+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a809bdd0 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.641 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.641+0000 7fd49affd640 1 -- 172.21.15.135:0/384670265 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4a400bc60 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.642+0000 7fd49affd640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd48c06c8f0 0x7fd48c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:31.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.642+0000 7fd49affd640 1 -- 172.21.15.135:0/384670265 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fd4a408c1f0 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.642+0000 7fd4ae166640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd48c06c8f0 0x7fd48c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:31.642 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.642+0000 7fd4ae166640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd48c06c8f0 0x7fd48c06edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80b3e30 tx=0x7fd49c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:31.644 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.646+0000 7fd49affd640 1 -- 172.21.15.135:0/384670265 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4a809bdd0 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.859+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4a8068a20 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.858 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.859+0000 7fd49affd640 1 -- 172.21.15.135:0/384670265 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7fd4a405a490 con 0x7fd4a80cd2b0 2024-04-17T06:10:31.859 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:10:31.859 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1269381293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1269381293},{"type":"v1","addr":"172.21.15.190:6834","nonce":1269381293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4865,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/354919697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":354919697},{"type":"v1","addr":"172.21.15.190:6835","nonce":354919697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5376,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6836/1997531955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1997531955},{"type":"v1","addr":"172.21.15.135:6836","nonce":1997531955}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-17T06:10:27.119795+0000","modified":"2024-04-17T06:10:29.130787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5379":{"gid":5379,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6837/3309431937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6835","nonce":3309431937},{"type":"v1","addr":"172.21.15.135:6837","nonce":3309431937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5379,"qdb_cluster":[5379]},"id":2}]} 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:dumped fsmap epoch 16 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd48c06c8f0 msgr2=0x7fd48c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd48c06c8f0 0x7fd48c06edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80b3e30 tx=0x7fd49c009040 comp rx=0 tx=0).stop 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 msgr2=0x7fd4a8155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a8155c90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400d800 tx=0x7fd4a400dcc0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 shutdown_connections 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4a81601f0 0x7fd4a80ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fd48c06c8f0 0x7fd48c06edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4a80cd2b0 0x7fd4a8155c90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 --2- 172.21.15.135:0/384670265 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4a809b100 0x7fd4a80cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:31.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 >> 172.21.15.135:0/384670265 conn(0x7fd4a80b4190 msgr2=0x7fd4a81605d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:31.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 shutdown_connections 2024-04-17T06:10:31.861 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:31.861+0000 7fd4b03ca640 1 -- 172.21.15.135:0/384670265 wait complete. 2024-04-17T06:10:32.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-17T06:10:27.119795+0000', 'modified': '2024-04-17T06:10:29.130787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5379}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5379': {'gid': 5379, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6837/3309431937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6835', 'nonce': 3309431937}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 3309431937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5379, 'qdb_cluster': [5379]} 2024-04-17T06:10:32.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-17T06:10:32.275 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-17T06:10:32.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.383+0000 7f9e51959640 1 Processor -- start 2024-04-17T06:10:32.382 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.383+0000 7f9e51959640 1 -- start start 2024-04-17T06:10:32.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.384+0000 7f9e51959640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c15ed00 0x7f9e4c0d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.384+0000 7f9e51959640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9e4c09bd50 0x7f9e4c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.384+0000 7f9e51959640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4c0d13f0 con 0x7f9e4c09af30 2024-04-17T06:10:32.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.384+0000 7f9e51959640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f9e4c0d19b0 con 0x7f9e4c09bd50 2024-04-17T06:10:32.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.384+0000 7f9e51959640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c0d16d0 con 0x7f9e4c15ed00 2024-04-17T06:10:32.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4a7fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9e4c09bd50 0x7f9e4c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c15ed00 0x7f9e4c0d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4a7fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9e4c09bd50 0x7f9e4c09a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:50834/0 (socket says 172.21.15.135:50834) 2024-04-17T06:10:32.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c15ed00 0x7f9e4c0d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:44608/0 (socket says 172.21.15.135:44608) 2024-04-17T06:10:32.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4a7fc640 1 -- 172.21.15.135:0/1981594884 learned_addr learned my addr 172.21.15.135:0/1981594884 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:32.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4a7fc640 1 -- 172.21.15.135:0/1981594884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c15ed00 msgr2=0x7f9e4c0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/1981594884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c15ed00 0x7f9e4c0d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4a7fc640 1 -- 172.21.15.135:0/1981594884 >> v1:172.21.15.190:6789/0 conn(0x7f9e4c09af30 legacy=0x7f9e4c0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.385+0000 7f9e4a7fc640 1 -- 172.21.15.135:0/1981594884 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c06cc20 con 0x7f9e4c09bd50 2024-04-17T06:10:32.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/1981594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 0x7f9e4c09a9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009870 tx=0x7f9e4002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fbd8b26eafff2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e49ffb640 1 -- 172.21.15.135:0/1981594884 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40034070 con 0x7f9e4c09bd50 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e49ffb640 1 -- 172.21.15.135:0/1981594884 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e40030840 con 0x7f9e4c09bd50 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e49ffb640 1 -- 172.21.15.135:0/1981594884 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40030c10 con 0x7f9e4c09bd50 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e51959640 1 -- 172.21.15.135:0/1981594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 msgr2=0x7f9e4c09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e51959640 1 --2- 172.21.15.135:0/1981594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 0x7f9e4c09a9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009870 tx=0x7f9e4002b980 comp rx=0 tx=0).stop 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e51959640 1 -- 172.21.15.135:0/1981594884 shutdown_connections 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e51959640 1 --2- 172.21.15.135:0/1981594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 0x7f9e4c09a9f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e51959640 1 --2- 172.21.15.135:0/1981594884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c15ed00 0x7f9e4c0d0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.386+0000 7f9e51959640 1 -- 172.21.15.135:0/1981594884 >> 172.21.15.135:0/1981594884 conn(0x7f9e4c0b4190 msgr2=0x7f9e4c0b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.387+0000 7f9e51959640 1 -- 172.21.15.135:0/1981594884 shutdown_connections 2024-04-17T06:10:32.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.387+0000 7f9e51959640 1 -- 172.21.15.135:0/1981594884 wait complete. 2024-04-17T06:10:32.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.388+0000 7f9e51959640 1 Processor -- start 2024-04-17T06:10:32.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.388+0000 7f9e51959640 1 -- start start 2024-04-17T06:10:32.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e51959640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c09af30 0x7f9e4c158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e51959640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 0x7f9e4c159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.388 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e51959640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e4c15ed00 0x7f9e4c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e51959640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c06e430 con 0x7f9e4c15ed00 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e51959640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c0d10e0 con 0x7f9e4c09bd50 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e51959640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c06e5b0 con 0x7f9e4c09af30 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e4c15ed00 0x7f9e4c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e4c15ed00 0x7f9e4c0cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54840/0 (socket says 172.21.15.135:54840) 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 -- 172.21.15.135:0/3115504131 learned_addr learned my addr 172.21.15.135:0/3115504131 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c09af30 0x7f9e4c158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 0x7f9e4c159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 -- 172.21.15.135:0/3115504131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c09af30 msgr2=0x7f9e4c158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c09af30 0x7f9e4c158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 -- 172.21.15.135:0/3115504131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 msgr2=0x7f9e4c159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 0x7f9e4c159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.389 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 -- 172.21.15.135:0/3115504131 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c1f43c0 con 0x7f9e4c15ed00 2024-04-17T06:10:32.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e4b7fe640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e4c15ed00 0x7f9e4c0cb780 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c6b0 tx=0x7f9e4400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e33fff640 1 -- 172.21.15.135:0/3115504131 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44003b70 con 0x7f9e4c15ed00 2024-04-17T06:10:32.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e33fff640 1 -- 172.21.15.135:0/3115504131 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44003d30 con 0x7f9e4c15ed00 2024-04-17T06:10:32.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e51959640 1 -- 172.21.15.135:0/3115504131 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c159ea0 con 0x7f9e4c15ed00 2024-04-17T06:10:32.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e33fff640 1 -- 172.21.15.135:0/3115504131 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44005310 con 0x7f9e4c15ed00 2024-04-17T06:10:32.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.389+0000 7f9e51959640 1 -- 172.21.15.135:0/3115504131 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c15a460 con 0x7f9e4c15ed00 2024-04-17T06:10:32.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.391+0000 7f9e33fff640 1 -- 172.21.15.135:0/3115504131 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e440054b0 con 0x7f9e4c15ed00 2024-04-17T06:10:32.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.392+0000 7f9e33fff640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9e3806c8f0 0x7f9e3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.392+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9e3806c8f0 0x7f9e3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.392+0000 7f9e33fff640 1 -- 172.21.15.135:0/3115504131 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f9e4408ba80 con 0x7f9e4c15ed00 2024-04-17T06:10:32.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.392+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9e3806c8f0 0x7f9e3806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009b40 tx=0x7f9e3c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.393+0000 7f9e54624740 1 Processor -- start 2024-04-17T06:10:32.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.393+0000 7f9e54624740 1 -- start start 2024-04-17T06:10:32.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.393+0000 7f9e54624740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.393+0000 7f9e54624740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e380a1f0 0x5597e380cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.393+0000 7f9e54624740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e37dd2f0 0x5597e380d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.393+0000 7f9e54624740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597e37479c0 con 0x5597e3809e10 2024-04-17T06:10:32.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.393+0000 7f9e54624740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5597e36ee650 con 0x5597e380a1f0 2024-04-17T06:10:32.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.393+0000 7f9e54624740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5597e3743980 con 0x5597e37dd2f0 2024-04-17T06:10:32.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37dcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54850/0 (socket says 172.21.15.135:54850) 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 -- 172.21.15.135:0/566969907 learned_addr learned my addr 172.21.15.135:0/566969907 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e380a1f0 0x5597e380cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4b7fe640 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e37dd2f0 0x5597e380d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 -- 172.21.15.135:0/566969907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e37dd2f0 msgr2=0x5597e380d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e37dd2f0 0x5597e380d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 -- 172.21.15.135:0/566969907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e380a1f0 msgr2=0x5597e380cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e380a1f0 0x5597e380cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 -- 172.21.15.135:0/566969907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e345a2d0 con 0x5597e3809e10 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e4affd640 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37dcdb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c011ee0 tx=0x7f9e3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b8c5fcc2bab979 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e317fa640 1 -- 172.21.15.135:0/566969907 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c01b070 con 0x5597e3809e10 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e317fa640 1 -- 172.21.15.135:0/566969907 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3c027070 con 0x5597e3809e10 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.394+0000 7f9e317fa640 1 -- 172.21.15.135:0/566969907 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c003a40 con 0x5597e3809e10 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 -- 172.21.15.135:0/566969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 msgr2=0x5597e37dcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37dcdb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c011ee0 tx=0x7f9e3c014000 comp rx=0 tx=0).stop 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 -- 172.21.15.135:0/566969907 shutdown_connections 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e37dd2f0 0x5597e380d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e380a1f0 0x5597e380cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 --2- 172.21.15.135:0/566969907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37dcdb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 -- 172.21.15.135:0/566969907 >> 172.21.15.135:0/566969907 conn(0x5597e37df020 msgr2=0x5597e3827ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 -- 172.21.15.135:0/566969907 shutdown_connections 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.395+0000 7f9e54624740 1 -- 172.21.15.135:0/566969907 wait complete. 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 1 -- start start 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 1 Processor -- start 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 0x5597e380f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e3822650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597e37e15c0 con 0x5597e3809e10 2024-04-17T06:10:32.400 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5597e3701a10 con 0x5597e38221d0 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5597e37e09b0 con 0x5597e380a1f0 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.396+0000 7f9e54624740 10 client.0 _ll_register_callbacks cb 0x5597e373d250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37f7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54864/0 (socket says 172.21.15.135:54864) 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 -- 172.21.15.135:0/3597354047 learned_addr learned my addr 172.21.15.135:0/3597354047 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 0x5597e380f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4b7fe640 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e3822650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 -- 172.21.15.135:0/3597354047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 msgr2=0x5597e380f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 0x5597e380f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 -- 172.21.15.135:0/3597354047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 msgr2=0x5597e3822650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e3822650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 -- 172.21.15.135:0/3597354047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e35246b0 con 0x5597e3809e10 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37f7d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c024040 tx=0x7f9e3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde2d9a278f4e708 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.401 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3597354047 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c01b040 con 0x5597e3809e10 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3597354047 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f9e3c036c70 con 0x5597e3809e10 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e54624740 1 -- 172.21.15.135:0/3597354047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e3c027070 con 0x5597e3809e10 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3597354047 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f9e3c03d050 con 0x5597e3809e10 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e54624740 10 client.5562 fetch_fsmap learned FSMap version 16 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.397+0000 7f9e54624740 1 -- 172.21.15.135:0/3597354047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5597e3810320 con 0x5597e3809e10 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3597354047 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3c027070 con 0x5597e3809e10 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3597354047 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f9e3c040020 con 0x5597e3809e10 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e54624740 10 client.5562 fetch_fsmap finished waiting for FSMap version 16 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e54624740 10 client.5562 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e54624740 20 client.5562 populate_metadata read hostname 'smithi135' 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e54624740 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5597e37facf0 0x5597e3846330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e54624740 4 client.5562 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:10:32.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e54624740 1 -- 172.21.15.135:0/3597354047 --> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597e37553e0 con 0x5597e37facf0 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5597e37facf0 0x5597e3846330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e2a7fc640 10 client.5562 ms_handle_connect on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.398+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5597e37facf0 0x5597e3846330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27433152bbac73d server_cookie=c2cf54b068496d2 in_seq=0 out_seq=0 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3597354047 <== mds.0 v2:172.21.15.135:6835/3309431937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597e37553e0 con 0x5597e37facf0 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e2a7fc640 10 client.5562 handle_command_reply: tid=0 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 client.5562 shutdown 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 -- 172.21.15.135:0/3597354047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 msgr2=0x5597e37f7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37f7d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c024040 tx=0x7f9e3c004730 comp rx=0 tx=0).stop 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 -- 172.21.15.135:0/3597354047 shutdown_connections 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5597e37facf0 0x5597e3846330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e3822650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e2a7fc640 0 client.5562 ms_handle_reset on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 0x5597e380f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 --2- 172.21.15.135:0/3597354047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e37f7d50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 -- 172.21.15.135:0/3597354047 >> 172.21.15.135:0/3597354047 conn(0x5597e37dd2f0 msgr2=0x5597e380cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:32.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 -- 172.21.15.135:0/3597354047 shutdown_connections 2024-04-17T06:10:32.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 1 -- 172.21.15.135:0/3597354047 wait complete. 2024-04-17T06:10:32.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.399+0000 7f9e54624740 20 client.5562 trim_cache size 0 max 16384 2024-04-17T06:10:32.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.417+0000 7f9e54624740 1 Processor -- start 2024-04-17T06:10:32.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.417+0000 7f9e54624740 1 -- start start 2024-04-17T06:10:32.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e54624740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e3809e10 0x5597e380f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e54624740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e380a1f0 0x5597e380db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e54624740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e380e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e54624740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c201840 con 0x5597e380a1f0 2024-04-17T06:10:32.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e54624740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c200840 con 0x5597e38221d0 2024-04-17T06:10:32.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e54624740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c1ffbd0 con 0x5597e3809e10 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e380a1f0 0x5597e380db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e380a1f0 0x5597e380db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54872/0 (socket says 172.21.15.135:54872) 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 -- 172.21.15.135:0/3100546447 learned_addr learned my addr 172.21.15.135:0/3100546447 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 -- 172.21.15.135:0/3100546447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e3809e10 msgr2=0x5597e380f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3100546447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e3809e10 0x5597e380f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 -- 172.21.15.135:0/3100546447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 msgr2=0x5597e380e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3100546447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e380e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 -- 172.21.15.135:0/3100546447 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e3810bd0 con 0x5597e380a1f0 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3100546447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e380a1f0 0x5597e380db90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9e4002b9c0 tx=0x7f9e40008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b52c49960891734 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e30ff9640 1 -- 172.21.15.135:0/3100546447 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40034040 con 0x5597e380a1f0 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e30ff9640 1 -- 172.21.15.135:0/3100546447 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e40030bf0 con 0x5597e380a1f0 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.418+0000 7f9e30ff9640 1 -- 172.21.15.135:0/3100546447 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40050480 con 0x5597e380a1f0 2024-04-17T06:10:32.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 -- 172.21.15.135:0/3100546447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e380a1f0 msgr2=0x5597e380db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 --2- 172.21.15.135:0/3100546447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e380a1f0 0x5597e380db90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9e4002b9c0 tx=0x7f9e40008030 comp rx=0 tx=0).stop 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 -- 172.21.15.135:0/3100546447 shutdown_connections 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 --2- 172.21.15.135:0/3100546447 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e380e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 --2- 172.21.15.135:0/3100546447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e380a1f0 0x5597e380db90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 --2- 172.21.15.135:0/3100546447 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e3809e10 0x5597e380f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 -- 172.21.15.135:0/3100546447 >> 172.21.15.135:0/3100546447 conn(0x5597e37dd2f0 msgr2=0x5597e37f7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 -- 172.21.15.135:0/3100546447 shutdown_connections 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 -- 172.21.15.135:0/3100546447 wait complete. 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.419+0000 7f9e54624740 1 -- start start 2024-04-17T06:10:32.419 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e54624740 1 Processor -- start 2024-04-17T06:10:32.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e54624740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e380cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e54624740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 0x5597e380f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.423 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e54624740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e38a0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e54624740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597e3810600 con 0x5597e3809e10 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e54624740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c2001f0 con 0x5597e38221d0 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e54624740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5597e37f9d90 con 0x5597e380a1f0 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e54624740 10 client.0 _ll_register_callbacks cb 0x7f9e400096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e380cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e380cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54882/0 (socket says 172.21.15.135:54882) 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.420+0000 7f9e4affd640 1 -- 172.21.15.135:0/3290951218 learned_addr learned my addr 172.21.15.135:0/3290951218 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 0x5597e380f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4b7fe640 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e38a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4affd640 1 -- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 msgr2=0x5597e380f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 0x5597e380f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4affd640 1 -- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 msgr2=0x5597e38a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e38a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.424 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4affd640 1 -- 172.21.15.135:0/3290951218 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e380faf0 con 0x5597e3809e10 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4b7fe640 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e38a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4affd640 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e380cf30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c011690 tx=0x7f9e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305d9297dd10dcd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3290951218 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c045070 con 0x5597e3809e10 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3290951218 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f9e3c022880 con 0x5597e3809e10 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 1 -- 172.21.15.135:0/3290951218 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e40030bf0 con 0x5597e3809e10 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3290951218 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f9e3c050050 con 0x5597e3809e10 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 10 client.5568 fetch_fsmap learned FSMap version 16 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 1 -- 172.21.15.135:0/3290951218 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5597e3841d00 con 0x5597e3809e10 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3290951218 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e40030bf0 con 0x5597e3809e10 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3290951218 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f9e3c053020 con 0x5597e3809e10 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 10 client.5568 fetch_fsmap finished waiting for FSMap version 16 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 10 client.5568 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-17T06:10:32.425 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 20 client.5568 populate_metadata read hostname 'smithi135' 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5597e380e330 0x5597e38a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 4 client.5568 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e54624740 1 -- 172.21.15.135:0/3290951218 --> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] -- command(tid 0: {"prefix": "status"}) -- 0x5597e374f250 con 0x5597e380e330 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.421+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5597e380e330 0x5597e38a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e2a7fc640 10 client.5568 ms_handle_connect on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e4a7fc640 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5597e380e330 0x5597e38a3bb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1dd8cec492b8c18 server_cookie=f91657a86f5461d8 in_seq=0 out_seq=0 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e2a7fc640 1 -- 172.21.15.135:0/3290951218 <== mds.0 v2:172.21.15.135:6835/3309431937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5597e374f250 con 0x5597e380e330 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e2a7fc640 10 client.5568 handle_command_reply: tid=0 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 client.5568 shutdown 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 -- 172.21.15.135:0/3290951218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 msgr2=0x5597e380cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e380cf30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c011690 tx=0x7f9e3c0046c0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 -- 172.21.15.135:0/3290951218 shutdown_connections 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5597e380e330 0x5597e38a3bb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.426 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5597e38221d0 0x5597e38a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5597e380a1f0 0x5597e380f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e2a7fc640 0 client.5568 ms_handle_reset on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 --2- 172.21.15.135:0/3290951218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597e3809e10 0x5597e380cf30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 -- 172.21.15.135:0/3290951218 >> 172.21.15.135:0/3290951218 conn(0x5597e37dd2f0 msgr2=0x5597e37f7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 -- 172.21.15.135:0/3290951218 shutdown_connections 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 1 -- 172.21.15.135:0/3290951218 wait complete. 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.422+0000 7f9e54624740 20 client.5568 trim_cache size 0 max 16384 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stdout: "cluster_fsid": "96728bf8-42ba-4aa8-88ba-489677bf3f41", 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stdout: "whoami": 0, 2024-04-17T06:10:32.427 INFO:teuthology.orchestra.run.smithi135.stdout: "id": 5379, 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout: "want_state": "up:active", 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout: "state": "up:active", 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "cephfs", 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout: "rank_uptime": 4.6562445659999998, 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout: "mdsmap_epoch": 16, 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch": 38, 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_epoch_barrier": 38, 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout: "uptime": 11.530149709 2024-04-17T06:10:32.428 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-17T06:10:32.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 -- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9e3806c8f0 msgr2=0x7f9e3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9e3806c8f0 0x7f9e3806edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009b40 tx=0x7f9e3c009190 comp rx=0 tx=0).stop 2024-04-17T06:10:32.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 -- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e4c15ed00 msgr2=0x7f9e4c0cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e4c15ed00 0x7f9e4c0cb780 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c6b0 tx=0x7f9e4400cb70 comp rx=0 tx=0).stop 2024-04-17T06:10:32.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 -- 172.21.15.135:0/3115504131 shutdown_connections 2024-04-17T06:10:32.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9e3806c8f0 0x7f9e3806edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e4c15ed00 0x7f9e4c0cb780 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4c09bd50 0x7f9e4c159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 --2- 172.21.15.135:0/3115504131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e4c09af30 0x7f9e4c158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 -- 172.21.15.135:0/3115504131 >> 172.21.15.135:0/3115504131 conn(0x7f9e4c0b4190 msgr2=0x7f9e4c0711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:32.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 -- 172.21.15.135:0/3115504131 shutdown_connections 2024-04-17T06:10:32.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.423+0000 7f9e51959640 1 -- 172.21.15.135:0/3115504131 wait complete. 2024-04-17T06:10:32.432 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T06:10:32.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.537+0000 7f090139f640 1 Processor -- start 2024-04-17T06:10:32.535 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.537+0000 7f090139f640 1 -- start start 2024-04-17T06:10:32.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.537+0000 7f090139f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08fc06d4b0 0x7f08fc06d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.537+0000 7f090139f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc0d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.537+0000 7f090139f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08fc06ddd0 con 0x7f08fc06caa0 2024-04-17T06:10:32.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.537+0000 7f090139f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f08fc0cd490 con 0x7f08fc06d4b0 2024-04-17T06:10:32.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.537+0000 7f090139f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc0ce930 con 0x7f08fc15ead0 2024-04-17T06:10:32.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc0d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.537 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fbfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08fc06d4b0 0x7f08fc06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc0d1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:44666/0 (socket says 172.21.15.135:44666) 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fbfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f08fc06d4b0 0x7f08fc06d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.135:50886/0 (socket says 172.21.15.135:50886) 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f0900b9e640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f08fc06caa0 0x7f08fc0b75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.135:35132/0 (socket says 172.21.15.135:35132) 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fb7fe640 1 -- 172.21.15.135:0/448836850 learned_addr learned my addr 172.21.15.135:0/448836850 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fb7fe640 1 -- 172.21.15.135:0/448836850 >> v2:172.21.15.190:3300/0 conn(0x7f08fc06d4b0 msgr2=0x7f08fc06d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fb7fe640 1 --2- 172.21.15.135:0/448836850 >> v2:172.21.15.190:3300/0 conn(0x7f08fc06d4b0 0x7f08fc06d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fb7fe640 1 -- 172.21.15.135:0/448836850 >> v1:172.21.15.190:6789/0 conn(0x7f08fc06caa0 legacy=0x7f08fc0b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fb7fe640 1 -- 172.21.15.135:0/448836850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc0d1a10 con 0x7f08fc15ead0 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.538+0000 7f08fbfff640 1 --2- 172.21.15.135:0/448836850 >> v2:172.21.15.190:3300/0 conn(0x7f08fc06d4b0 0x7f08fc06d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.539+0000 7f08fb7fe640 1 --2- 172.21.15.135:0/448836850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc0d1170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009870 tx=0x7f08f002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad506d09410dde91 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.539+0000 7f08faffd640 1 -- 172.21.15.135:0/448836850 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0034070 con 0x7f08fc15ead0 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.539+0000 7f08faffd640 1 -- 172.21.15.135:0/448836850 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f0030840 con 0x7f08fc15ead0 2024-04-17T06:10:32.538 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.539+0000 7f08faffd640 1 -- 172.21.15.135:0/448836850 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0030c10 con 0x7f08fc15ead0 2024-04-17T06:10:32.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.540+0000 7f090139f640 1 -- 172.21.15.135:0/448836850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 msgr2=0x7f08fc0d1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.540+0000 7f090139f640 1 --2- 172.21.15.135:0/448836850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc0d1170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009870 tx=0x7f08f002b980 comp rx=0 tx=0).stop 2024-04-17T06:10:32.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.540+0000 7f090139f640 1 -- 172.21.15.135:0/448836850 shutdown_connections 2024-04-17T06:10:32.541 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.540+0000 7f090139f640 1 --2- 172.21.15.135:0/448836850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc0d1170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.540+0000 7f090139f640 1 --2- 172.21.15.135:0/448836850 >> v2:172.21.15.190:3300/0 conn(0x7f08fc06d4b0 0x7f08fc06d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.540+0000 7f090139f640 1 -- 172.21.15.135:0/448836850 >> 172.21.15.135:0/448836850 conn(0x7f08fc0b3f60 msgr2=0x7f08fc0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.540+0000 7f090139f640 1 -- 172.21.15.135:0/448836850 shutdown_connections 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.540+0000 7f090139f640 1 -- 172.21.15.135:0/448836850 wait complete. 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.541+0000 7f090139f640 1 Processor -- start 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.541+0000 7f090139f640 1 -- start start 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f090139f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08fc06caa0 0x7f08fc155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f090139f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08fc06d4b0 0x7f08fc0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f090139f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f090139f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0cd120 con 0x7f08fc06caa0 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f090139f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0cce40 con 0x7f08fc06d4b0 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f090139f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc0d16b0 con 0x7f08fc15ead0 2024-04-17T06:10:32.542 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08fc06caa0 0x7f08fc155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08fc06caa0 0x7f08fc155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54894/0 (socket says 172.21.15.135:54894) 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 -- 172.21.15.135:0/2364810242 learned_addr learned my addr 172.21.15.135:0/2364810242 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fb7fe640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08fc06d4b0 0x7f08fc0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f0900b9e640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 -- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 msgr2=0x7f08fc152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 -- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08fc06d4b0 msgr2=0x7f08fc0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08fc06d4b0 0x7f08fc0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 -- 172.21.15.135:0/2364810242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc0cc020 con 0x7f08fc06caa0 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fb7fe640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08fc06d4b0 0x7f08fc0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.542+0000 7f08fbfff640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08fc06caa0 0x7f08fc155ac0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00ebe0 tx=0x7f08ec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.543+0000 7f08f8ff9640 1 -- 172.21.15.135:0/2364810242 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec017070 con 0x7f08fc06caa0 2024-04-17T06:10:32.543 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.543+0000 7f08f8ff9640 1 -- 172.21.15.135:0/2364810242 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec00cc30 con 0x7f08fc06caa0 2024-04-17T06:10:32.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.543+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc0af380 con 0x7f08fc06caa0 2024-04-17T06:10:32.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.543+0000 7f08f8ff9640 1 -- 172.21.15.135:0/2364810242 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec013470 con 0x7f08fc06caa0 2024-04-17T06:10:32.544 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.543+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc0cb2e0 con 0x7f08fc06caa0 2024-04-17T06:10:32.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.544+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08fc0af860 con 0x7f08fc06caa0 2024-04-17T06:10:32.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.544+0000 7f08f8ff9640 1 -- 172.21.15.135:0/2364810242 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08ec01a070 con 0x7f08fc06caa0 2024-04-17T06:10:32.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.545+0000 7f08f8ff9640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f08e806c8f0 0x7f08e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:32.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.545+0000 7f08f8ff9640 1 -- 172.21.15.135:0/2364810242 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f08ec08bcc0 con 0x7f08fc06caa0 2024-04-17T06:10:32.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.545+0000 7f08fb7fe640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f08e806c8f0 0x7f08e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:32.545 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.545+0000 7f08fb7fe640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f08e806c8f0 0x7f08e806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f08f0036000 tx=0x7f08f003b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:32.547 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.549+0000 7f08f8ff9640 1 -- 172.21.15.135:0/2364810242 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08fc0cc020 con 0x7f08fc06caa0 2024-04-17T06:10:32.748 INFO:tasks.ceph.mds.a.smithi135.stderr:2024-04-17T06:10:32.750+0000 7f2a53748640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-17T06:10:32.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.761+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f08fc0cb720 con 0x7f08fc06caa0 2024-04-17T06:10:32.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.761+0000 7f08f8ff9640 1 -- 172.21.15.135:0/2364810242 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f08ec059f70 con 0x7f08fc06caa0 2024-04-17T06:10:32.759 INFO:teuthology.orchestra.run.smithi135.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T06:10:32.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f08e806c8f0 msgr2=0x7f08e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f08e806c8f0 0x7f08e806edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f08f0036000 tx=0x7f08f003b040 comp rx=0 tx=0).stop 2024-04-17T06:10:32.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08fc06caa0 msgr2=0x7f08fc155ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08fc06caa0 0x7f08fc155ac0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00ebe0 tx=0x7f08ec00c520 comp rx=0 tx=0).stop 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 shutdown_connections 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f08fc15ead0 0x7f08fc152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f08fc06d4b0 0x7f08fc0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f08e806c8f0 0x7f08e806edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 --2- 172.21.15.135:0/2364810242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08fc06caa0 0x7f08fc155ac0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 >> 172.21.15.135:0/2364810242 conn(0x7f08fc0b3f60 msgr2=0x7f08fc0b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 shutdown_connections 2024-04-17T06:10:32.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:32.763+0000 7f090139f640 1 -- 172.21.15.135:0/2364810242 wait complete. 2024-04-17T06:10:32.772 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-17T06:10:32.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T06:10:32.772 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi135.front.sepia.ceph.com 2024-04-17T06:10:32.773 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-17T06:10:32.773 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-17T06:10:32.773 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-17T06:10:32.773 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-17T06:10:32.773 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T06:10:32.773 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T06:10:32.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:32.773 DEBUG:teuthology.orchestra.run.smithi135:> ip addr 2024-04-17T06:10:32.789 INFO:teuthology.orchestra.run.smithi135.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 ::1/128 scope host 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:76 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f0 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:d9:36:77 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: altname enp7s0f1 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:82 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:32.790 INFO:teuthology.orchestra.run.smithi135.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 0c:c4:7a:88:80:83 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: inet 172.21.15.135/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft 42688sec preferred_lft 42688sec 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::ec4:7aff:fe88:8083/64 scope link 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 4e:db:16:ca:7f:a9 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::c2:8aff:fe91:6349/64 scope link 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether 4e:db:16:ca:7f:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-17T06:10:32.791 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::4cdb:16ff:feca:7fa9/64 scope link 2024-04-17T06:10:32.792 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:32.792 INFO:teuthology.orchestra.run.smithi135.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T06:10:32.792 INFO:teuthology.orchestra.run.smithi135.stdout: link/ether e2:37:9c:2f:d7:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-17T06:10:32.792 INFO:teuthology.orchestra.run.smithi135.stdout: inet6 fe80::e037:9cff:fe2f:d775/64 scope link 2024-04-17T06:10:32.792 INFO:teuthology.orchestra.run.smithi135.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:32.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:32.792 DEBUG:teuthology.orchestra.run.smithi135:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T06:10:32.857 INFO:teuthology.orchestra.run.smithi135.stdout:1 2024-04-17T06:10:32.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:32.859 DEBUG:teuthology.orchestra.run.smithi135:> route 2024-04-17T06:10:32.915 INFO:teuthology.orchestra.run.smithi135.stdout:Kernel IP routing table 2024-04-17T06:10:32.915 INFO:teuthology.orchestra.run.smithi135.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T06:10:32.915 INFO:teuthology.orchestra.run.smithi135.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T06:10:32.915 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T06:10:32.915 INFO:teuthology.orchestra.run.smithi135.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T06:10:32.916 DEBUG:teuthology.orchestra.run.smithi135:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:10:32.916 DEBUG:teuthology.orchestra.run.smithi135:> set -e 2024-04-17T06:10:32.916 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T06:10:32.916 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T06:10:32.916 DEBUG:teuthology.orchestra.run.smithi135:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T06:10:32.916 DEBUG:teuthology.orchestra.run.smithi135:> ') 2024-04-17T06:10:33.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:10:33.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:10:33.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:33.054 DEBUG:teuthology.orchestra.run.smithi135:> ip netns list 2024-04-17T06:10:33.070 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-17T06:10:33.070 INFO:teuthology.orchestra.run.smithi135.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-17T06:10:33.070 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-17T06:10:33.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:33.070 DEBUG:teuthology.orchestra.run.smithi135:> sudo ip link set brx.0 up 2024-04-17T06:10:33.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:10:33.139 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-17T06:10:33.194 INFO:teuthology.orchestra.run.smithi135.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-17T06:10:33.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:10:33.195 DEBUG:teuthology.orchestra.run.smithi135:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-17T06:10:33.250 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-17T06:10:33.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:33.251 DEBUG:teuthology.orchestra.run.smithi135:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-17T06:10:33.337 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-17T06:10:33.338 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-17T06:10:33.338 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:10:33.346 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-17T06:10:33.346 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-17T06:10:33.346 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:10:33.346 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:10:33.346 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:10:33.351 INFO:teuthology.orchestra.run.smithi135.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout:invalid new device string format 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: resolved to: "172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790" 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=96728bf8-42ba-4aa8-88ba-489677bf3f41" will pass to kernel 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-17T06:10:33.352 INFO:teuthology.orchestra.run.smithi135.stdout: You just mounted a file system that supports labels which does not 2024-04-17T06:10:33.353 INFO:teuthology.orchestra.run.smithi135.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-17T06:10:33.353 INFO:teuthology.orchestra.run.smithi135.stdout: applications will generate AVC messages and not be allowed access to 2024-04-17T06:10:33.353 INFO:teuthology.orchestra.run.smithi135.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-17T06:10:33.354 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-17T06:10:33.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:33.354 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-17T06:10:33.383 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:10:33.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:10:33.478 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:10:33.484 INFO:teuthology.orchestra.run.smithi135.stdout:client5574 2024-04-17T06:10:33.484 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:33.485 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:33.501 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-17T06:10:33.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.648+0000 7f6928e77640 1 Processor -- start 2024-04-17T06:10:33.646 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.648+0000 7f6928e77640 1 -- start start 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6928e77640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240d3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6928e77640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f69240b83b0 0x7f69240d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6928e77640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69240d2b60 con 0x7f69240d3b40 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6928e77640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f69240d21a0 con 0x7f69240b83b0 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6928e77640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f69240d1ec0 con 0x7f692415eb80 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f692259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6922d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f69240b83b0 0x7f69240d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f692259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240d3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.135:44696/0 (socket says 172.21.15.135:44696) 2024-04-17T06:10:33.647 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f692259c640 1 -- 172.21.15.135:0/3896795138 learned_addr learned my addr 172.21.15.135:0/3896795138 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:33.648 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6922d9d640 1 -- 172.21.15.135:0/3896795138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 msgr2=0x7f69240d3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:33.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6922d9d640 1 --2- 172.21.15.135:0/3896795138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240d3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6922d9d640 1 -- 172.21.15.135:0/3896795138 >> v1:172.21.15.190:6789/0 conn(0x7f69240d3b40 legacy=0x7f69240b7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:33.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f6922d9d640 1 -- 172.21.15.135:0/3896795138 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69240d2e00 con 0x7f69240b83b0 2024-04-17T06:10:33.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.649+0000 7f692259c640 1 --2- 172.21.15.135:0/3896795138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240d3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:10:33.649 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.650+0000 7f6922d9d640 1 --2- 172.21.15.135:0/3896795138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 0x7f69240d45c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f691c009d90 tx=0x7f691c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af9be29d8808db59 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.650+0000 7f692159a640 1 -- 172.21.15.135:0/3896795138 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c039070 con 0x7f69240b83b0 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.650+0000 7f692159a640 1 -- 172.21.15.135:0/3896795138 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c02cc50 con 0x7f69240b83b0 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f692159a640 1 -- 172.21.15.135:0/3896795138 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c034440 con 0x7f69240b83b0 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f6928e77640 1 -- 172.21.15.135:0/3896795138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 msgr2=0x7f69240d45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f6928e77640 1 --2- 172.21.15.135:0/3896795138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 0x7f69240d45c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f691c009d90 tx=0x7f691c02bea0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f6928e77640 1 -- 172.21.15.135:0/3896795138 shutdown_connections 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f6928e77640 1 --2- 172.21.15.135:0/3896795138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 0x7f69240d45c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f6928e77640 1 --2- 172.21.15.135:0/3896795138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240d3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f6928e77640 1 -- 172.21.15.135:0/3896795138 >> 172.21.15.135:0/3896795138 conn(0x7f6924073720 msgr2=0x7f692406cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f6928e77640 1 -- 172.21.15.135:0/3896795138 shutdown_connections 2024-04-17T06:10:33.650 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.651+0000 7f6928e77640 1 -- 172.21.15.135:0/3896795138 wait complete. 2024-04-17T06:10:33.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6928e77640 1 Processor -- start 2024-04-17T06:10:33.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6928e77640 1 -- start start 2024-04-17T06:10:33.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6928e77640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 0x7f69241608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:33.652 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6928e77640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69240d3b40 0x7f6924147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6928e77640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6928e77640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f692415ef60 con 0x7f69240d3b40 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6928e77640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f69240d4b00 con 0x7f69240b83b0 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6928e77640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f6924154bf0 con 0x7f692415eb80 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6921d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69240d3b40 0x7f6924147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6921d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69240d3b40 0x7f6924147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:54896/0 (socket says 172.21.15.135:54896) 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.653+0000 7f6921d9b640 1 -- 172.21.15.135:0/1046450990 learned_addr learned my addr 172.21.15.135:0/1046450990 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f692259c640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 0x7f69241608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6922d9d640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6921d9b640 1 -- 172.21.15.135:0/1046450990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 msgr2=0x7f69240c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6921d9b640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6921d9b640 1 -- 172.21.15.135:0/1046450990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 msgr2=0x7f69241608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6921d9b640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 0x7f69241608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.653 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6921d9b640 1 -- 172.21.15.135:0/1046450990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6924160ea0 con 0x7f69240d3b40 2024-04-17T06:10:33.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6921d9b640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69240d3b40 0x7f6924147230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f691800c430 tx=0x7f691800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:33.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6912ffd640 1 -- 172.21.15.135:0/1046450990 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918017070 con 0x7f69240d3b40 2024-04-17T06:10:33.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6924161180 con 0x7f69240d3b40 2024-04-17T06:10:33.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6912ffd640 1 -- 172.21.15.135:0/1046450990 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69180052d0 con 0x7f69240d3b40 2024-04-17T06:10:33.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6912ffd640 1 -- 172.21.15.135:0/1046450990 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6918003850 con 0x7f69240d3b40 2024-04-17T06:10:33.654 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.654+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6924161740 con 0x7f69240d3b40 2024-04-17T06:10:33.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.655+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6924074940 con 0x7f69240d3b40 2024-04-17T06:10:33.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.656+0000 7f6912ffd640 1 -- 172.21.15.135:0/1046450990 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f691801a070 con 0x7f69240d3b40 2024-04-17T06:10:33.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.656+0000 7f6912ffd640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:33.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.656+0000 7f6912ffd640 1 -- 172.21.15.135:0/1046450990 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f691808b060 con 0x7f69240d3b40 2024-04-17T06:10:33.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.656+0000 7f692259c640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:33.655 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.657+0000 7f692259c640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f690406c8f0 0x7f690406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6914006ca0 tx=0x7f6914009240 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:33.658 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.660+0000 7f6912ffd640 1 -- 172.21.15.135:0/1046450990 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6918058c30 con 0x7f69240d3b40 2024-04-17T06:10:33.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.884+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f69240d4780 con 0x7f69240d3b40 2024-04-17T06:10:33.882 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.884+0000 7f6912ffd640 1 -- 172.21.15.135:0/1046450990 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f691805cf30 con 0x7f69240d3b40 2024-04-17T06:10:33.882 INFO:teuthology.orchestra.run.smithi135.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-17T06:10:33.884 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f690406c8f0 msgr2=0x7f690406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f690406c8f0 0x7f690406edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6914006ca0 tx=0x7f6914009240 comp rx=0 tx=0).stop 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69240d3b40 msgr2=0x7f6924147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69240d3b40 0x7f6924147230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f691800c430 tx=0x7f691800c8f0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 shutdown_connections 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f692415eb80 0x7f69240c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f690406c8f0 0x7f690406edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69240d3b40 0x7f6924147230 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 --2- 172.21.15.135:0/1046450990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69240b83b0 0x7f69241608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.886+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 >> 172.21.15.135:0/1046450990 conn(0x7f6924073720 msgr2=0x7f692406cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.887+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 shutdown_connections 2024-04-17T06:10:33.885 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:33.887+0000 7f6928e77640 1 -- 172.21.15.135:0/1046450990 wait complete. 2024-04-17T06:10:33.895 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-17T06:10:33.895 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-17T06:10:33.896 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2024-04-17T06:10:33.896 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-17T06:10:33.896 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-17T06:10:33.896 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-17T06:10:33.896 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-17T06:10:33.896 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-17T06:10:33.896 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-17T06:10:33.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:33.896 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2024-04-17T06:10:33.916 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2024-04-17T06:10:33.917 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42668sec preferred_lft 42668sec 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 66:92:72:a8:56:7f brd ff:ff:ff:ff:ff:ff 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::4857:30ff:fe97:155/64 scope link 2024-04-17T06:10:33.918 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run.smithi190.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 66:92:72:a8:56:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::6492:72ff:fea8:567f/64 scope link 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run.smithi190.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 92:9b:5b:41:db:cb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::909b:5bff:fe41:dbcb/64 scope link 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2024-04-17T06:10:33.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:33.919 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-17T06:10:33.989 INFO:teuthology.orchestra.run.smithi190.stdout:1 2024-04-17T06:10:33.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:33.991 DEBUG:teuthology.orchestra.run.smithi190:> route 2024-04-17T06:10:34.051 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2024-04-17T06:10:34.052 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-17T06:10:34.052 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-17T06:10:34.052 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-17T06:10:34.052 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-17T06:10:34.052 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-17T06:10:34.052 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2024-04-17T06:10:34.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-17T06:10:34.052 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-17T06:10:34.053 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-17T06:10:34.053 DEBUG:teuthology.orchestra.run.smithi190:> ') 2024-04-17T06:10:34.146 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:10:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-17T06:10:34.193 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:10:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:10:34.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:34.199 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2024-04-17T06:10:34.215 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-17T06:10:34.215 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-17T06:10:34.215 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-17T06:10:34.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:34.215 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2024-04-17T06:10:34.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:10:34.284 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-17T06:10:34.340 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-17T06:10:34.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-17T06:10:34.340 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-17T06:10:34.395 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-17T06:10:34.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:34.395 DEBUG:teuthology.orchestra.run.smithi190:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-17T06:10:34.483 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-17T06:10:34.483 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-17T06:10:34.483 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:10:34.492 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-17T06:10:34.492 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-17T06:10:34.492 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:10:34.492 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:10:34.492 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:10:34.497 INFO:teuthology.orchestra.run.smithi190.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-17T06:10:34.497 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-17T06:10:34.497 INFO:teuthology.orchestra.run.smithi190.stdout:invalid new device string format 2024-04-17T06:10:34.497 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:3300,172.21.15.190:3300,172.21.15.190:3301" 2024-04-17T06:10:34.497 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-17T06:10:34.497 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: resolved to: "172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790" 2024-04-17T06:10:34.497 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.135:6789,172.21.15.190:6789,172.21.15.190:6790:/ 2024-04-17T06:10:34.497 INFO:teuthology.orchestra.run.smithi190.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=96728bf8-42ba-4aa8-88ba-489677bf3f41" will pass to kernel 2024-04-17T06:10:34.498 INFO:teuthology.orchestra.run.smithi190.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-17T06:10:34.498 INFO:teuthology.orchestra.run.smithi190.stdout: You just mounted a file system that supports labels which does not 2024-04-17T06:10:34.498 INFO:teuthology.orchestra.run.smithi190.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-17T06:10:34.498 INFO:teuthology.orchestra.run.smithi190.stdout: applications will generate AVC messages and not be allowed access to 2024-04-17T06:10:34.498 INFO:teuthology.orchestra.run.smithi190.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-17T06:10:34.500 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-17T06:10:34.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:10:34.500 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-17T06:10:34.528 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:10:34.618 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:10:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:10:34.620 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:10:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:10:34.625 INFO:teuthology.orchestra.run.smithi190.stdout:client4943 2024-04-17T06:10:34.626 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:34.626 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:44.643 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-04-17T06:10:44.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.753+0000 7f827de4a640 1 Processor -- start 2024-04-17T06:10:44.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.753+0000 7f827de4a640 1 -- start start 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.753+0000 7f827de4a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f827815ec00 0x7f82780d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.753+0000 7f827de4a640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82780d43d0 0x7f82780d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f827de4a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82780d3330 con 0x7f82780d1810 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f827de4a640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f827815efe0 con 0x7f82780d43d0 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f827de4a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f82780d3f40 con 0x7f827815ec00 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f8276ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82780d43d0 0x7f82780d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f8276ffd640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f82780d43d0 0x7f82780d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51480/0 (socket says 172.21.15.135:51480) 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f8276ffd640 1 -- 172.21.15.135:0/3496053350 learned_addr learned my addr 172.21.15.135:0/3496053350 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f82777fe640 1 --2- 172.21.15.135:0/3496053350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f827815ec00 0x7f82780d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f8276ffd640 1 -- 172.21.15.135:0/3496053350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f827815ec00 msgr2=0x7f82780d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f8276ffd640 1 --2- 172.21.15.135:0/3496053350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f827815ec00 0x7f82780d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f8276ffd640 1 -- 172.21.15.135:0/3496053350 >> v1:172.21.15.190:6789/0 conn(0x7f82780d1810 legacy=0x7f82780730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.754+0000 7f8276ffd640 1 -- 172.21.15.135:0/3496053350 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82780d3610 con 0x7f82780d43d0 2024-04-17T06:10:44.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.755+0000 7f8276ffd640 1 --2- 172.21.15.135:0/3496053350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d43d0 0x7f82780d12d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8260009870 tx=0x7f826002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f98d70098346dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:44.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.755+0000 7f82767fc640 1 -- 172.21.15.135:0/3496053350 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260034070 con 0x7f82780d43d0 2024-04-17T06:10:44.761 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.755+0000 7f82767fc640 1 -- 172.21.15.135:0/3496053350 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8260030840 con 0x7f82780d43d0 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.755+0000 7f82767fc640 1 -- 172.21.15.135:0/3496053350 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260030c10 con 0x7f82780d43d0 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.755+0000 7f827de4a640 1 -- 172.21.15.135:0/3496053350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d43d0 msgr2=0x7f82780d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.755+0000 7f827de4a640 1 --2- 172.21.15.135:0/3496053350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d43d0 0x7f82780d12d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8260009870 tx=0x7f826002b980 comp rx=0 tx=0).stop 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.756+0000 7f827de4a640 1 -- 172.21.15.135:0/3496053350 shutdown_connections 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.756+0000 7f827de4a640 1 --2- 172.21.15.135:0/3496053350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d43d0 0x7f82780d12d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.756+0000 7f827de4a640 1 --2- 172.21.15.135:0/3496053350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f827815ec00 0x7f82780d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.756+0000 7f827de4a640 1 -- 172.21.15.135:0/3496053350 >> 172.21.15.135:0/3496053350 conn(0x7f827806cc90 msgr2=0x7f827806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.756+0000 7f827de4a640 1 -- 172.21.15.135:0/3496053350 shutdown_connections 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.756+0000 7f827de4a640 1 -- 172.21.15.135:0/3496053350 wait complete. 2024-04-17T06:10:44.762 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.757+0000 7f827de4a640 1 Processor -- start 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.757+0000 7f827de4a640 1 -- start start 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f827de4a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d1810 0x7f82780c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f827de4a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82780d43d0 0x7f82781f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f827de4a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f827815ec00 0x7f8278147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f827de4a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82780d3bd0 con 0x7f82780d1810 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f827de4a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82780d1c70 con 0x7f827815ec00 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f827de4a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f82780d2ed0 con 0x7f82780d43d0 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d1810 0x7f82780c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d1810 0x7f82780c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51496/0 (socket says 172.21.15.135:51496) 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 -- 172.21.15.135:0/916040557 learned_addr learned my addr 172.21.15.135:0/916040557 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f8276ffd640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82780d43d0 0x7f82781f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f8277fff640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f827815ec00 0x7f8278147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 -- 172.21.15.135:0/916040557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82780d43d0 msgr2=0x7f82781f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82780d43d0 0x7f82781f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 -- 172.21.15.135:0/916040557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f827815ec00 msgr2=0x7f8278147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f827815ec00 0x7f8278147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 -- 172.21.15.135:0/916040557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82781f83c0 con 0x7f82780d1810 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f82777fe640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d1810 0x7f82780c8de0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f826800d620 tx=0x7f826800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f826ffff640 1 -- 172.21.15.135:0/916040557 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826800b620 con 0x7f82780d1810 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f826ffff640 1 -- 172.21.15.135:0/916040557 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8268019070 con 0x7f82780d1810 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f826ffff640 1 -- 172.21.15.135:0/916040557 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826800ba00 con 0x7f82780d1810 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f827de4a640 1 -- 172.21.15.135:0/916040557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82781f86a0 con 0x7f82780d1810 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.758+0000 7f827de4a640 1 -- 172.21.15.135:0/916040557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82780c88c0 con 0x7f82780d1810 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.760+0000 7f826ffff640 1 -- 172.21.15.135:0/916040557 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f826800bba0 con 0x7f82780d1810 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.760+0000 7f826ffff640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f825806c8f0 0x7f825806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.760+0000 7f826ffff640 1 -- 172.21.15.135:0/916040557 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f826808bfc0 con 0x7f82780d1810 2024-04-17T06:10:44.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.760+0000 7f8276ffd640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f825806c8f0 0x7f825806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.761+0000 7f8276ffd640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f825806c8f0 0x7f825806edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8260036000 tx=0x7f8260008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.761+0000 7f8280a34740 1 Processor -- start 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.761+0000 7f8280a34740 1 -- start start 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f8280a34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedc03e0 0x5634ced93380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f8280a34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc07c0 0x5634cedc3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f8280a34740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634ced938c0 0x5634cedc36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f8280a34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5634ced5b030 con 0x5634cedc03e0 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f8280a34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5634cecfe100 con 0x5634cedc07c0 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f8280a34740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5634cec9b570 con 0x5634ced938c0 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f82777fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedc03e0 0x5634ced93380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f82777fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedc03e0 0x5634ced93380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51510/0 (socket says 172.21.15.135:51510) 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f82777fe640 1 -- 172.21.15.135:0/1175127743 learned_addr learned my addr 172.21.15.135:0/1175127743 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f8277fff640 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634ced938c0 0x5634cedc36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.772 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.762+0000 7f8276ffd640 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc07c0 0x5634cedc3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f82777fe640 1 -- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634ced938c0 msgr2=0x5634cedc36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f82777fe640 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634ced938c0 0x5634cedc36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f82777fe640 1 -- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc07c0 msgr2=0x5634cedc3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f82777fe640 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc07c0 0x5634cedc3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f82777fe640 1 -- 172.21.15.135:0/1175127743 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634cecb6ed0 con 0x5634cedc03e0 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f8276ffd640 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc07c0 0x5634cedc3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f8277fff640 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634ced938c0 0x5634cedc36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f82777fe640 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedc03e0 0x5634ced93380 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f826808a2d0 tx=0x7f8268059c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09dc41ee8f9a261 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f826d7fa640 1 -- 172.21.15.135:0/1175127743 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268072020 con 0x5634cedc03e0 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f826d7fa640 1 -- 172.21.15.135:0/1175127743 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8268062740 con 0x5634cedc03e0 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f826d7fa640 1 -- 172.21.15.135:0/1175127743 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268056280 con 0x5634cedc03e0 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f8280a34740 1 -- 172.21.15.135:0/1175127743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedc03e0 msgr2=0x5634ced93380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.763+0000 7f8280a34740 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedc03e0 0x5634ced93380 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f826808a2d0 tx=0x7f8268059c10 comp rx=0 tx=0).stop 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 -- 172.21.15.135:0/1175127743 shutdown_connections 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634ced938c0 0x5634cedc36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.773 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc07c0 0x5634cedc3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 --2- 172.21.15.135:0/1175127743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedc03e0 0x5634ced93380 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 -- 172.21.15.135:0/1175127743 >> 172.21.15.135:0/1175127743 conn(0x5634ced955f0 msgr2=0x5634cedde170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 -- 172.21.15.135:0/1175127743 shutdown_connections 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 -- 172.21.15.135:0/1175127743 wait complete. 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 -- start start 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.764+0000 7f8280a34740 1 Processor -- start 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8280a34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cedf28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8280a34740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8280a34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8280a34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5634cecb7e80 con 0x5634cedf1d10 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8280a34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5634cecfa0c0 con 0x5634cedc03e0 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8280a34740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5634ced97c90 con 0x5634cedc07c0 2024-04-17T06:10:44.774 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8280a34740 10 client.0 _ll_register_callbacks cb 0x5634cecf4560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51518/0 (socket says 172.21.15.135:51518) 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 -- 172.21.15.135:0/102873398 learned_addr learned my addr 172.21.15.135:0/102873398 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8276ffd640 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f82777fe640 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cedf28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 -- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 msgr2=0x5634cedee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedee140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 -- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 msgr2=0x5634cedf28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cedf28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 -- 172.21.15.135:0/102873398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634cea103f0 con 0x5634cedf1d10 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f82777fe640 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cedf28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.765+0000 7f8277fff640 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf2190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f82700116a0 tx=0x7f8270011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabdf8cdd13e745a server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f82567fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f82567fc640 1 -- 172.21.15.135:0/102873398 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270019070 con 0x5634cedf1d10 2024-04-17T06:10:44.775 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f82567fc640 1 -- 172.21.15.135:0/102873398 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8270006b50 con 0x5634cedf1d10 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 1 -- 172.21.15.135:0/102873398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8268062740 con 0x5634cedf1d10 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f82567fc640 1 -- 172.21.15.135:0/102873398 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f8270024050 con 0x5634cedf1d10 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 10 client.5589 fetch_fsmap learned FSMap version 16 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 1 -- 172.21.15.135:0/102873398 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5634ceada7d0 con 0x5634cedf1d10 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f82567fc640 1 -- 172.21.15.135:0/102873398 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8268062740 con 0x5634cedf1d10 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f82567fc640 1 -- 172.21.15.135:0/102873398 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f8270027020 con 0x5634cedf1d10 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 10 client.5589 fetch_fsmap finished waiting for FSMap version 16 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 10 client.5589 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 20 client.5589 populate_metadata read hostname 'smithi135' 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5634cedc60a0 0x5634ceddc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 4 client.5589 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f8280a34740 1 -- 172.21.15.135:0/102873398 --> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634ceca3f30 con 0x5634cedc60a0 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f82777fe640 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5634cedc60a0 0x5634ceddc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.766+0000 7f82777fe640 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5634cedc60a0 0x5634ceddc540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c6e54d72d5138cc server_cookie=63a053bbd732d522 in_seq=0 out_seq=0 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f82567fc640 10 client.5589 ms_handle_connect on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:44.776 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f82567fc640 1 -- 172.21.15.135:0/102873398 <== mds.0 v2:172.21.15.135:6835/3309431937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634ceca3f30 con 0x5634cedc60a0 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f82567fc640 10 client.5589 handle_command_reply: tid=0 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 client.5589 shutdown 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 -- 172.21.15.135:0/102873398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 msgr2=0x5634cedf2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf2190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f82700116a0 tx=0x7f8270011b60 comp rx=0 tx=0).stop 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 -- 172.21.15.135:0/102873398 shutdown_connections 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf2190 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5634cedc60a0 0x5634ceddc540 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 --2- 172.21.15.135:0/102873398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cedf28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f82567fc640 0 client.5589 ms_handle_reset on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 -- 172.21.15.135:0/102873398 >> 172.21.15.135:0/102873398 conn(0x5634ced938c0 msgr2=0x5634ced92f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 -- 172.21.15.135:0/102873398 shutdown_connections 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.767+0000 7f8280a34740 1 -- 172.21.15.135:0/102873398 wait complete. 2024-04-17T06:10:44.777 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.768+0000 7f8280a34740 20 client.5589 trim_cache size 0 max 16384 2024-04-17T06:10:44.779 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8280a34740 1 Processor -- start 2024-04-17T06:10:44.780 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8280a34740 1 -- start start 2024-04-17T06:10:44.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8280a34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634ced92a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8280a34740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedf07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8280a34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.781 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8280a34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8278201690 con 0x5634cedf1d10 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8280a34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82781f4e80 con 0x5634cedc03e0 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8280a34740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8278161fb0 con 0x5634cedc07c0 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8277fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8277fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51530/0 (socket says 172.21.15.135:51530) 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.781+0000 7f8277fff640 1 -- 172.21.15.135:0/993454674 learned_addr learned my addr 172.21.15.135:0/993454674 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f82777fe640 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634ced92a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f8276ffd640 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedf07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f8277fff640 1 -- 172.21.15.135:0/993454674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 msgr2=0x5634cedf07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f8277fff640 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedf07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f8277fff640 1 -- 172.21.15.135:0/993454674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 msgr2=0x5634ced92a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f8277fff640 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634ced92a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f8277fff640 1 -- 172.21.15.135:0/993454674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634ceddf330 con 0x5634cedf1d10 2024-04-17T06:10:44.782 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f8277fff640 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf0d20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8270005720 tx=0x7f827000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2b323b153165a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:44.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f826cff9640 1 -- 172.21.15.135:0/993454674 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270019040 con 0x5634cedf1d10 2024-04-17T06:10:44.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f826cff9640 1 -- 172.21.15.135:0/993454674 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8270011110 con 0x5634cedf1d10 2024-04-17T06:10:44.783 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.782+0000 7f826cff9640 1 -- 172.21.15.135:0/993454674 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827000f7a0 con 0x5634cedf1d10 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 -- 172.21.15.135:0/993454674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 msgr2=0x5634cedf0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf0d20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8270005720 tx=0x7f827000a000 comp rx=0 tx=0).stop 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 -- 172.21.15.135:0/993454674 shutdown_connections 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedf0d20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedf07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 --2- 172.21.15.135:0/993454674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634ced92a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 -- 172.21.15.135:0/993454674 >> 172.21.15.135:0/993454674 conn(0x5634ced938c0 msgr2=0x5634ceddb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 -- 172.21.15.135:0/993454674 shutdown_connections 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 -- 172.21.15.135:0/993454674 wait complete. 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.783+0000 7f8280a34740 1 -- start start 2024-04-17T06:10:44.788 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8280a34740 1 Processor -- start 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8280a34740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cee57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8280a34740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedc2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8280a34740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedc30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8280a34740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5634ced951b0 con 0x5634cedf1d10 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8280a34740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f82781f4830 con 0x5634cedc03e0 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8280a34740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5634cedf1a00 con 0x5634cedc07c0 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8280a34740 10 client.0 _ll_register_callbacks cb 0x7f82700050d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8277fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedc30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.784+0000 7f8277fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedc30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51538/0 (socket says 172.21.15.135:51538) 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8277fff640 1 -- 172.21.15.135:0/3655974636 learned_addr learned my addr 172.21.15.135:0/3655974636 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f82777fe640 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cee57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8276ffd640 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedc2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8277fff640 1 -- 172.21.15.135:0/3655974636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 msgr2=0x5634cedc2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8277fff640 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedc2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8277fff640 1 -- 172.21.15.135:0/3655974636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 msgr2=0x5634cee57500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:44.789 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8277fff640 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cee57500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8277fff640 1 -- 172.21.15.135:0/3655974636 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634cedc84f0 con 0x5634cedf1d10 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8277fff640 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedc30f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8270021010 tx=0x7f8270003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96219a5e97ebb0b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f82567fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f82567fc640 1 -- 172.21.15.135:0/3655974636 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270002030 con 0x5634cedf1d10 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f82567fc640 1 -- 172.21.15.135:0/3655974636 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f8270032c70 con 0x5634cedf1d10 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8280a34740 1 -- 172.21.15.135:0/3655974636 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8270011110 con 0x5634cedf1d10 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f82567fc640 1 -- 172.21.15.135:0/3655974636 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f8270032e70 con 0x5634cedf1d10 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8280a34740 10 client.5595 fetch_fsmap learned FSMap version 16 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.785+0000 7f8280a34740 1 -- 172.21.15.135:0/3655974636 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5634cee59160 con 0x5634cedf1d10 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f82567fc640 1 -- 172.21.15.135:0/3655974636 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8270011110 con 0x5634cedf1d10 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f82567fc640 1 -- 172.21.15.135:0/3655974636 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f8270027070 con 0x5634cedf1d10 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f8280a34740 10 client.5595 fetch_fsmap finished waiting for FSMap version 16 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f8280a34740 10 client.5595 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T06:10:44.790 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f8280a34740 20 client.5595 populate_metadata read hostname 'smithi135' 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f8280a34740 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5634cedb2c80 0x5634cedf07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f8280a34740 4 client.5595 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f8280a34740 1 -- 172.21.15.135:0/3655974636 --> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] -- command(tid 0: {"prefix": "counter dump"}) -- 0x5634ced0bb20 con 0x5634cedb2c80 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f82777fe640 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5634cedb2c80 0x5634cedf07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f82567fc640 10 client.5595 ms_handle_connect on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.786+0000 7f82777fe640 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5634cedb2c80 0x5634cedf07e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79cabec9f43e4990 server_cookie=1343d89c49750827 in_seq=0 out_seq=0 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f82567fc640 1 -- 172.21.15.135:0/3655974636 <== mds.0 v2:172.21.15.135:6835/3309431937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+26832 (crc 0 0 0) 0x5634ced0bb20 con 0x5634cedb2c80 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f82567fc640 10 client.5595 handle_command_reply: tid=0 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 client.5595 shutdown 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 -- 172.21.15.135:0/3655974636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 msgr2=0x5634cedc30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedc30f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8270021010 tx=0x7f8270003ae0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 -- 172.21.15.135:0/3655974636 shutdown_connections 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5634cedf1d10 0x5634cedc30f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5634cedc07c0 0x5634cedc2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5634cedb2c80 0x5634cedf07e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.791 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 --2- 172.21.15.135:0/3655974636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5634cedc03e0 0x5634cee57500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 -- 172.21.15.135:0/3655974636 >> 172.21.15.135:0/3655974636 conn(0x5634ced938c0 msgr2=0x5634ceddb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:44.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f82567fc640 0 client.5595 ms_handle_reset on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:44.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 -- 172.21.15.135:0/3655974636 shutdown_connections 2024-04-17T06:10:44.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 1 -- 172.21.15.135:0/3655974636 wait complete. 2024-04-17T06:10:44.792 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.787+0000 7f8280a34740 20 client.5595 trim_cache size 0 max 16384 2024-04-17T06:10:44.792 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker": [ 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "0" 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "1" 2024-04-17T06:10:44.793 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "2" 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:10:44.794 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": [ 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 39, 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 31, 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 11491, 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 20848, 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 8, 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.005343083, 2024-04-17T06:10:44.795 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000732939, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.002380390, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000961878, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 30, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.179583827, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.005986127 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 5, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000001687, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000337 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:10:44.796 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": [ 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 17, 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 21, 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 4732, 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 121806, 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 8, 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 5, 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.009683830, 2024-04-17T06:10:44.797 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000609660, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.008669870, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000296004, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 20, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.130167834, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.006508391 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000135, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000067 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.798 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 93808 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": [ 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 104, 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 90, 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 126510, 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 52763, 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 7, 2024-04-17T06:10:44.799 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.008057909, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.002395142, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.012600180, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000880521, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 90, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.028455634, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000316173 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 28, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000006959, 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000248 2024-04-17T06:10:44.800 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 113424, 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 12368 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": [ 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": [ 2024-04-17T06:10:44.801 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 52, 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.001413877, 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000027189 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": [ 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.802 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000015348, 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000007674 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": [ 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.803 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 28, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 28, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 28, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.003560238, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000127151 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 12, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 11, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 0, 2024-04-17T06:10:44.804 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 11, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 10, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 1, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 2, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 2, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 53, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 51, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-04-17T06:10:44.805 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-04-17T06:10:44.806 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-04-17T06:10:44.807 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 1 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": [ 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-04-17T06:10:44.808 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-04-17T06:10:44.809 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_metrics": [ 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "cephfs", 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "5379" 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.810 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "num_clients": 2 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_metrics-cephfs": [ 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4943", 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "rank": "0" 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_hits": 24, 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_miss": 0, 2024-04-17T06:10:44.811 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_read_latency": 0.000000000, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_write_latency": 0.000000000, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_metadata_latency": 0.000000000, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_hits": 0, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_miss": 0, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_files": 0, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_inodes": 1, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "pinned_icaps": 1, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "total_inodes": 1, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_ops": 0, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_size": 0, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_ops": 0, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_size": 0 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.812 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5574", 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "rank": "0" 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_hits": 5, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_miss": 19, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_read_latency": 0.000000000, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_write_latency": 0.000000000, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_metadata_latency": 0.000000000, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_hits": 0, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_miss": 0, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_files": 0, 2024-04-17T06:10:44.813 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_inodes": 1, 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "pinned_icaps": 1, 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "total_inodes": 1, 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_ops": 0, 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_size": 0, 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_ops": 0, 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_size": 0 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": [ 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.814 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 6, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 6, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 2, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 2, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 2, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-04-17T06:10:44.815 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 4196743, 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3, 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.009711252, 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003237084 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": [ 2024-04-17T06:10:44.816 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 13, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 13, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 0, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 12, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 12, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 0, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 11, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 11, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 0, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 2, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 2, 2024-04-17T06:10:44.817 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 42356, 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 231784 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": [ 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 28, 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 28, 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 2, 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-04-17T06:10:44.818 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 25, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.003090088, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000123603 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.819 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-04-17T06:10:44.820 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.821 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.822 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.823 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000203295, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000101647 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.824 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.825 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.826 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000266855, 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000266855 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.827 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.828 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.829 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": [ 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 2, 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 2, 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-04-17T06:10:44.830 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 2, 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 27, 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 13, 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 6, 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": [ 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-04-17T06:10:44.831 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:10:44.832 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:10:44.833 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 232969, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 91, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:10:44.834 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 5424, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 76, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 53784, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 218, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-04-17T06:10:44.835 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": [ 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 41, 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 13559, 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 37, 2024-04-17T06:10:44.836 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 41, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 41, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.831982741, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.020292261 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 41, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 65 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 41, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 0, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 41, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-04-17T06:10:44.837 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 11, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 3, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 7, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 6, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-04-17T06:10:44.838 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 22, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-04-17T06:10:44.839 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-04-17T06:10:44.840 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 38, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 1, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 8, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 8, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 15, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 1 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.841 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": [ 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": [ 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.842 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": [ 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-04-17T06:10:44.843 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 154, 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 125220, 2024-04-17T06:10:44.844 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 154, 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 154, 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 125220, 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.845 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": [ 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:10:44.846 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 41, 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 13559, 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 41, 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 13559, 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": [ 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.847 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 41, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 41, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 41, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 41, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.848 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": [ 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:10:44.849 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.850 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x56392f628500": [ 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 6, 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 2439, 2024-04-17T06:10:44.851 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 6, 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3, 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 2439, 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-17T06:10:44.852 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-17T06:10:44.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 -- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f825806c8f0 msgr2=0x7f825806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f825806c8f0 0x7f825806edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8260036000 tx=0x7f8260008030 comp rx=0 tx=0).stop 2024-04-17T06:10:44.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 -- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d1810 msgr2=0x7f82780c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:44.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d1810 0x7f82780c8de0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f826800d620 tx=0x7f826800dae0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 -- 172.21.15.135:0/916040557 shutdown_connections 2024-04-17T06:10:44.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f827815ec00 0x7f8278147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82780d43d0 0x7f82781f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f825806c8f0 0x7f825806edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 --2- 172.21.15.135:0/916040557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82780d1810 0x7f82780c8de0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:44.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 -- 172.21.15.135:0/916040557 >> 172.21.15.135:0/916040557 conn(0x7f827806cc90 msgr2=0x7f82780b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:44.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 -- 172.21.15.135:0/916040557 shutdown_connections 2024-04-17T06:10:44.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44.788+0000 7f827de4a640 1 -- 172.21.15.135:0/916040557 wait complete. 2024-04-17T06:10:44.856 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:44.856 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:44.872 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:10:44.962 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:10:44.963 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:10:44.969 INFO:teuthology.orchestra.run.smithi135.stdout:client5574 2024-04-17T06:10:44.970 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:44.970 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:44.990 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-17T06:10:45.086 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:10:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-17T06:10:45.088 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:10:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:10:45.094 INFO:teuthology.orchestra.run.smithi190.stdout:client4943 2024-04-17T06:10:45.094 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:10:45.094 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:45.111 DEBUG:teuthology.orchestra.run.smithi135:> stdin-killer -- python3 -c ' 2024-04-17T06:10:45.111 DEBUG:teuthology.orchestra.run.smithi135:> import os 2024-04-17T06:10:45.111 DEBUG:teuthology.orchestra.run.smithi135:> import uuid 2024-04-17T06:10:45.111 DEBUG:teuthology.orchestra.run.smithi135:> 2024-04-17T06:10:45.111 DEBUG:teuthology.orchestra.run.smithi135:> n = 1000 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> create_hard_links = False 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> if 0 > 0: 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> create_hard_links = True 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> path = "/home/ubuntu/cephtest/mnt.0/test_dir/test_file" 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> dpath = os.path.dirname(path) 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> fnameprefix = os.path.basename(path) 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> os.makedirs(dpath, exist_ok=True) 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> try: 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> for i in range(n): 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> with open(fpath, '"'"'w'"'"') as f: 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> f.write(f"{i}") 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> if True: 2024-04-17T06:10:45.112 DEBUG:teuthology.orchestra.run.smithi135:> f.flush() 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> os.fsync(f.fileno()) 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> if False: 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> os.unlink(fpath) 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> if False: 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> os.fsync(dirfd) 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> if create_hard_links: 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> for j in range(0): 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> if False: 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> os.fsync(dirfd) 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> finally: 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> os.close(dirfd) 2024-04-17T06:10:45.113 DEBUG:teuthology.orchestra.run.smithi135:> ' 2024-04-17T06:10:49.242 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:10:49.248 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-04-17T06:10:49.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.395+0000 7feef54ba640 1 Processor -- start 2024-04-17T06:10:49.394 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.395+0000 7feef54ba640 1 -- start start 2024-04-17T06:10:49.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feef54ba640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feef01601f0 0x7feef00d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feef54ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feef54ba640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7feef00d0300 0x7feef009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feef54ba640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7feef00cd790 con 0x7feef00d0300 2024-04-17T06:10:49.396 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feef54ba640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7feef00cda70 con 0x7feef01601f0 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feef54ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7feef00cdd50 con 0x7feef00cd2b0 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feeef7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7feef00d0300 0x7feef009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feeef7fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7feef00d0300 0x7feef009a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51550/0 (socket says 172.21.15.135:51550) 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feeef7fe640 1 -- 172.21.15.135:0/3628069976 learned_addr learned my addr 172.21.15.135:0/3628069976 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feee67fc640 1 --2- 172.21.15.135:0/3628069976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.396+0000 7feeeeffd640 1 --2- 172.21.15.135:0/3628069976 >> v2:172.21.15.190:3300/0 conn(0x7feef01601f0 0x7feef00d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeef7fe640 1 -- 172.21.15.135:0/3628069976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 msgr2=0x7feef00ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeef7fe640 1 --2- 172.21.15.135:0/3628069976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeef7fe640 1 -- 172.21.15.135:0/3628069976 >> v2:172.21.15.190:3300/0 conn(0x7feef01601f0 msgr2=0x7feef00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeef7fe640 1 --2- 172.21.15.135:0/3628069976 >> v2:172.21.15.190:3300/0 conn(0x7feef01601f0 0x7feef00d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeef7fe640 1 -- 172.21.15.135:0/3628069976 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef00ce030 con 0x7feef00d0300 2024-04-17T06:10:49.397 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feee67fc640 1 --2- 172.21.15.135:0/3628069976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeeeffd640 1 --2- 172.21.15.135:0/3628069976 >> v2:172.21.15.190:3300/0 conn(0x7feef01601f0 0x7feef00d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeef7fe640 1 --2- 172.21.15.135:0/3628069976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef009a9f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feef006ee40 tx=0x7feee802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb08533652ffbe4b server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeee7fc640 1 -- 172.21.15.135:0/3628069976 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee802cb00 con 0x7feef00d0300 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeee7fc640 1 -- 172.21.15.135:0/3628069976 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee802cca0 con 0x7feef00d0300 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.397+0000 7feeee7fc640 1 -- 172.21.15.135:0/3628069976 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8034610 con 0x7feef00d0300 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 -- 172.21.15.135:0/3628069976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 msgr2=0x7feef009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 --2- 172.21.15.135:0/3628069976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef009a9f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feef006ee40 tx=0x7feee802bf60 comp rx=0 tx=0).stop 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 -- 172.21.15.135:0/3628069976 shutdown_connections 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 --2- 172.21.15.135:0/3628069976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef009a9f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 --2- 172.21.15.135:0/3628069976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.398 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 --2- 172.21.15.135:0/3628069976 >> v2:172.21.15.190:3300/0 conn(0x7feef01601f0 0x7feef00d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 -- 172.21.15.135:0/3628069976 >> 172.21.15.135:0/3628069976 conn(0x7feef00b4190 msgr2=0x7feef00b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:49.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 -- 172.21.15.135:0/3628069976 shutdown_connections 2024-04-17T06:10:49.399 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.398+0000 7feef54ba640 1 -- 172.21.15.135:0/3628069976 wait complete. 2024-04-17T06:10:49.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.399+0000 7feef54ba640 1 Processor -- start 2024-04-17T06:10:49.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.399+0000 7feef54ba640 1 -- start start 2024-04-17T06:10:49.402 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feef54ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feef54ba640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feef54ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef01601f0 0x7feef0071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feef54ba640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feef009b0b0 con 0x7feef00d0300 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feef54ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feef009af30 con 0x7feef01601f0 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feef54ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7feef009b230 con 0x7feef00cd2b0 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feee67fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feee67fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef014d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51562/0 (socket says 172.21.15.135:51562) 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feee67fc640 1 -- 172.21.15.135:0/3924583566 learned_addr learned my addr 172.21.15.135:0/3924583566 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.400+0000 7feeef7fe640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef01601f0 0x7feef0071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feeeeffd640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee67fc640 1 -- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 msgr2=0x7feef00b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.403 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee67fc640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee67fc640 1 -- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef01601f0 msgr2=0x7feef0071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee67fc640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef01601f0 0x7feef0071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee67fc640 1 -- 172.21.15.135:0/3924583566 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef00b1280 con 0x7feef00d0300 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feeef7fe640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef01601f0 0x7feef0071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee67fc640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef014d470 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feed800e810 tx=0x7feed800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee7fff640 1 -- 172.21.15.135:0/3924583566 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed800f040 con 0x7feef00d0300 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee7fff640 1 -- 172.21.15.135:0/3924583566 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feed8004030 con 0x7feef00d0300 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feef54ba640 1 -- 172.21.15.135:0/3924583566 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef014d9b0 con 0x7feef00d0300 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feee7fff640 1 -- 172.21.15.135:0/3924583566 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8004400 con 0x7feef00d0300 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.401+0000 7feef54ba640 1 -- 172.21.15.135:0/3924583566 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01f61d0 con 0x7feef00d0300 2024-04-17T06:10:49.404 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.402+0000 7feee7fff640 1 -- 172.21.15.135:0/3924583566 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feed800c670 con 0x7feef00d0300 2024-04-17T06:10:49.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.403+0000 7feee7fff640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7feed006c8f0 0x7feed006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.403+0000 7feeeeffd640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7feed006c8f0 0x7feed006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.403+0000 7feee7fff640 1 -- 172.21.15.135:0/3924583566 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7feed808b9c0 con 0x7feef00d0300 2024-04-17T06:10:49.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.404+0000 7feeeeffd640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7feed006c8f0 0x7feed006edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feee0006fd0 tx=0x7feee0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:49.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.404+0000 7feef817e740 1 Processor -- start 2024-04-17T06:10:49.405 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.404+0000 7feef817e740 1 -- start start 2024-04-17T06:10:49.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feef817e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89de60 0x5600fc89bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feef817e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89e240 0x5600fc855490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feef817e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc87d7a0 0x5600fc8559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feef817e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5600fc818030 con 0x5600fc89de60 2024-04-17T06:10:49.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feef817e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5600fc7bb100 con 0x5600fc87d7a0 2024-04-17T06:10:49.412 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feef817e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5600fc758570 con 0x5600fc89e240 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89de60 0x5600fc89bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89de60 0x5600fc89bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51574/0 (socket says 172.21.15.135:51574) 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 -- 172.21.15.135:0/1701225459 learned_addr learned my addr 172.21.15.135:0/1701225459 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 -- 172.21.15.135:0/1701225459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89e240 msgr2=0x5600fc855490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 --2- 172.21.15.135:0/1701225459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89e240 0x5600fc855490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 -- 172.21.15.135:0/1701225459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc87d7a0 msgr2=0x5600fc8559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 --2- 172.21.15.135:0/1701225459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc87d7a0 0x5600fc8559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 -- 172.21.15.135:0/1701225459 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fc773ed0 con 0x5600fc89de60 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.405+0000 7feeeeffd640 1 --2- 172.21.15.135:0/1701225459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89de60 0x5600fc89bb20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feee000fa80 tx=0x7feee0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14336d1827b1274f server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feee4ff9640 1 -- 172.21.15.135:0/1701225459 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee001b050 con 0x5600fc89de60 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feee4ff9640 1 -- 172.21.15.135:0/1701225459 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee0025070 con 0x5600fc89de60 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feee4ff9640 1 -- 172.21.15.135:0/1701225459 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0014d60 con 0x5600fc89de60 2024-04-17T06:10:49.413 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 -- 172.21.15.135:0/1701225459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89de60 msgr2=0x5600fc89bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 --2- 172.21.15.135:0/1701225459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89de60 0x5600fc89bb20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feee000fa80 tx=0x7feee0012000 comp rx=0 tx=0).stop 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 -- 172.21.15.135:0/1701225459 shutdown_connections 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 --2- 172.21.15.135:0/1701225459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc87d7a0 0x5600fc8559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 --2- 172.21.15.135:0/1701225459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89e240 0x5600fc855490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 --2- 172.21.15.135:0/1701225459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89de60 0x5600fc89bb20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 -- 172.21.15.135:0/1701225459 >> 172.21.15.135:0/1701225459 conn(0x5600fc8525f0 msgr2=0x5600fc89b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 -- 172.21.15.135:0/1701225459 shutdown_connections 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.406+0000 7feef817e740 1 -- 172.21.15.135:0/1701225459 wait complete. 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.407+0000 7feef817e740 1 -- start start 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.407+0000 7feef817e740 1 Processor -- start 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feef817e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc864420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feef817e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 0x5600fc8957d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.414 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feef817e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc87e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feef817e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5600fc774e80 con 0x5600fc87e150 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feef817e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x5600fc7b70c0 con 0x5600fc89e240 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feef817e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5600fc850c20 con 0x5600fc89de60 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feef817e740 10 client.0 _ll_register_callbacks cb 0x5600fc7b1560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc87e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc87e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51582/0 (socket says 172.21.15.135:51582) 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 -- 172.21.15.135:0/3736639721 learned_addr learned my addr 172.21.15.135:0/3736639721 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeeeffd640 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc864420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feee67fc640 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 0x5600fc8957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 -- 172.21.15.135:0/3736639721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 msgr2=0x5600fc864420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc864420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 -- 172.21.15.135:0/3736639721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 msgr2=0x5600fc8957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 0x5600fc8957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 -- 172.21.15.135:0/3736639721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fc4cd3f0 con 0x5600fc87e150 2024-04-17T06:10:49.415 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feeef7fe640 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc87e5d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feee8009f40 tx=0x7feee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b3acfa0b74aab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.408+0000 7feecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feecdffb640 1 -- 172.21.15.135:0/3736639721 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8044070 con 0x5600fc87e150 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feecdffb640 1 -- 172.21.15.135:0/3736639721 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7feee8034b30 con 0x5600fc87e150 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 1 -- 172.21.15.135:0/3736639721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee0025070 con 0x5600fc87e150 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feecdffb640 1 -- 172.21.15.135:0/3736639721 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7feee804f050 con 0x5600fc87e150 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 10 client.5607 fetch_fsmap learned FSMap version 16 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 1 -- 172.21.15.135:0/3736639721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5600fc5977d0 con 0x5600fc87e150 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feecdffb640 1 -- 172.21.15.135:0/3736639721 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee0025070 con 0x5600fc87e150 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feecdffb640 1 -- 172.21.15.135:0/3736639721 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7feee8052020 con 0x5600fc87e150 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 10 client.5607 fetch_fsmap finished waiting for FSMap version 16 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 10 client.5607 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T06:10:49.416 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 20 client.5607 populate_metadata read hostname 'smithi135' 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5600fc867160 0x5600fc867570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 4 client.5607 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feef817e740 1 -- 172.21.15.135:0/3736639721 --> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600fc760f30 con 0x5600fc867160 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feeeeffd640 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5600fc867160 0x5600fc867570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feeeeffd640 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5600fc867160 0x5600fc867570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397d4daebcf9bfdb server_cookie=f0d5a086eee94532 in_seq=0 out_seq=0 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.409+0000 7feecdffb640 10 client.5607 ms_handle_connect on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feecdffb640 1 -- 172.21.15.135:0/3736639721 <== mds.0 v2:172.21.15.135:6835/3309431937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600fc760f30 con 0x5600fc867160 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feecdffb640 10 client.5607 handle_command_reply: tid=0 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 client.5607 shutdown 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 -- 172.21.15.135:0/3736639721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 msgr2=0x5600fc87e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc87e5d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7feee8009f40 tx=0x7feee8004a40 comp rx=0 tx=0).stop 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 -- 172.21.15.135:0/3736639721 shutdown_connections 2024-04-17T06:10:49.417 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc87e5d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5600fc867160 0x5600fc867570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 0x5600fc8957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 --2- 172.21.15.135:0/3736639721 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc864420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feecdffb640 0 client.5607 ms_handle_reset on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:49.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 -- 172.21.15.135:0/3736639721 >> 172.21.15.135:0/3736639721 conn(0x5600fc87d7a0 msgr2=0x5600fc84f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:49.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 -- 172.21.15.135:0/3736639721 shutdown_connections 2024-04-17T06:10:49.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 1 -- 172.21.15.135:0/3736639721 wait complete. 2024-04-17T06:10:49.418 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.410+0000 7feef817e740 20 client.5607 trim_cache size 0 max 16384 2024-04-17T06:10:49.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.423+0000 7feef817e740 1 Processor -- start 2024-04-17T06:10:49.421 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.423+0000 7feef817e740 1 -- start start 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feef817e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc853ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feef817e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc897c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feef817e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 0x5600fc878060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feef817e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feef01fd280 con 0x5600fc87e150 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feef817e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feef006e300 con 0x5600fc89e240 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feef817e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7feef00b61c0 con 0x5600fc89de60 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc853ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc853ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51594/0 (socket says 172.21.15.135:51594) 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeeeffd640 1 -- 172.21.15.135:0/2102137884 learned_addr learned my addr 172.21.15.135:0/2102137884 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:49.429 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeef7fe640 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 0x5600fc878060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feee67fc640 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc897c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeeeffd640 1 -- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 msgr2=0x5600fc897c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeeeffd640 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc897c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeeeffd640 1 -- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 msgr2=0x5600fc878060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeeeffd640 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 0x5600fc878060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feeeeffd640 1 -- 172.21.15.135:0/2102137884 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fc84c310 con 0x5600fc87e150 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.424+0000 7feee67fc640 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc897c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feeeeffd640 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc853ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5600fc853580 tx=0x7feee0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a50f4474867f249 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feecffff640 1 -- 172.21.15.135:0/2102137884 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee003b070 con 0x5600fc87e150 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feecffff640 1 -- 172.21.15.135:0/2102137884 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee0025040 con 0x5600fc87e150 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feecffff640 1 -- 172.21.15.135:0/2102137884 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee001b040 con 0x5600fc87e150 2024-04-17T06:10:49.430 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 -- 172.21.15.135:0/2102137884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 msgr2=0x5600fc853ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc853ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5600fc853580 tx=0x7feee0014000 comp rx=0 tx=0).stop 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 -- 172.21.15.135:0/2102137884 shutdown_connections 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89e240 0x5600fc878060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc89de60 0x5600fc897c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 --2- 172.21.15.135:0/2102137884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc87e150 0x5600fc853ee0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 -- 172.21.15.135:0/2102137884 >> 172.21.15.135:0/2102137884 conn(0x5600fc87d7a0 msgr2=0x5600fc864230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 -- 172.21.15.135:0/2102137884 shutdown_connections 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.425+0000 7feef817e740 1 -- 172.21.15.135:0/2102137884 wait complete. 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.426+0000 7feef817e740 1 -- start start 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.426+0000 7feef817e740 1 Processor -- start 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feef817e740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc87e150 0x5600fc850570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feef817e740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89de60 0x5600fc91a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.431 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feef817e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89e240 0x5600fc8c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feef817e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5600fc867540 con 0x5600fc89e240 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feef817e740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7feef006dcb0 con 0x5600fc89de60 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feef817e740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x5600fc8c45d0 con 0x5600fc87e150 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feef817e740 10 client.0 _ll_register_callbacks cb 0x7feee8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89e240 0x5600fc8c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89e240 0x5600fc8c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:51604/0 (socket says 172.21.15.135:51604) 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 -- 172.21.15.135:0/1346460959 learned_addr learned my addr 172.21.15.135:0/1346460959 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feee67fc640 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89de60 0x5600fc91a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeeeffd640 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc87e150 0x5600fc850570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 -- 172.21.15.135:0/1346460959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc87e150 msgr2=0x5600fc850570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.432 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc87e150 0x5600fc850570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 -- 172.21.15.135:0/1346460959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89de60 msgr2=0x5600fc91a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:10:49.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89de60 0x5600fc91a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 -- 172.21.15.135:0/1346460959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fc87e6d0 con 0x5600fc89e240 2024-04-17T06:10:49.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feeef7fe640 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89e240 0x5600fc8c2f40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x5600fc884810 tx=0x7feee8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b1d8094ef86cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:10:49.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:10:49.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feecdffb640 1 -- 172.21.15.135:0/1346460959 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8057070 con 0x5600fc89e240 2024-04-17T06:10:49.433 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feecdffb640 1 -- 172.21.15.135:0/1346460959 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7feee8003990 con 0x5600fc89e240 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.427+0000 7feef817e740 1 -- 172.21.15.135:0/1346460959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee0025040 con 0x5600fc89e240 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feecdffb640 1 -- 172.21.15.135:0/1346460959 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7feee8061050 con 0x5600fc89e240 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feef817e740 10 client.5613 fetch_fsmap learned FSMap version 16 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feef817e740 1 -- 172.21.15.135:0/1346460959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5600fc8c3650 con 0x5600fc89e240 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feecdffb640 1 -- 172.21.15.135:0/1346460959 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee0025040 con 0x5600fc89e240 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feecdffb640 1 -- 172.21.15.135:0/1346460959 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7feee8064020 con 0x5600fc89e240 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feef817e740 10 client.5613 fetch_fsmap finished waiting for FSMap version 16 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feef817e740 10 client.5613 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feef817e740 20 client.5613 populate_metadata read hostname 'smithi135' 2024-04-17T06:10:49.434 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feef817e740 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5600fc87ebb0 0x5600fc87efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feef817e740 4 client.5613 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feef817e740 1 -- 172.21.15.135:0/1346460959 --> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] -- command(tid 0: {"prefix": "counter dump"}) -- 0x5600fc87fbb0 con 0x5600fc87ebb0 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feeeeffd640 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5600fc87ebb0 0x5600fc87efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feecdffb640 10 client.5613 ms_handle_connect on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.428+0000 7feeeeffd640 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5600fc87ebb0 0x5600fc87efc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3bc6a86defca594 server_cookie=4f737f2f91796a53 in_seq=0 out_seq=0 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feecdffb640 1 -- 172.21.15.135:0/1346460959 <== mds.0 v2:172.21.15.135:6835/3309431937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+27002 (crc 0 0 0) 0x5600fc87fbb0 con 0x5600fc87ebb0 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feecdffb640 10 client.5613 handle_command_reply: tid=0 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 client.5613 shutdown 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 -- 172.21.15.135:0/1346460959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89e240 msgr2=0x5600fc8c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89e240 0x5600fc8c2f40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x5600fc884810 tx=0x7feee8030000 comp rx=0 tx=0).stop 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 -- 172.21.15.135:0/1346460959 shutdown_connections 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600fc89e240 0x5600fc8c2f40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.435 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x5600fc87ebb0 0x5600fc87efc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5600fc89de60 0x5600fc91a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 --2- 172.21.15.135:0/1346460959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5600fc87e150 0x5600fc850570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feecdffb640 0 client.5613 ms_handle_reset on v2:172.21.15.135:6835/3309431937 2024-04-17T06:10:49.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 -- 172.21.15.135:0/1346460959 >> 172.21.15.135:0/1346460959 conn(0x5600fc87d7a0 msgr2=0x5600fc878740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:49.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 -- 172.21.15.135:0/1346460959 shutdown_connections 2024-04-17T06:10:49.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 1 -- 172.21.15.135:0/1346460959 wait complete. 2024-04-17T06:10:49.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.429+0000 7feef817e740 20 client.5613 trim_cache size 0 max 16384 2024-04-17T06:10:49.436 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker": [ 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "0" 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.437 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "1" 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "2" 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.438 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": [ 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.439 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1052, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1040, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 261553, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 4593017, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 8, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.067432176, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.027746161, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.014825483, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.019321469, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1039, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.196905749, 2024-04-17T06:10:49.440 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000189514 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 8, 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000002131, 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000266 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.441 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": [ 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 19, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 24, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 5024, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 173888, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 10, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 7, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.010362998, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.000722315, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.009168575, 2024-04-17T06:10:49.442 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000296004, 2024-04-17T06:10:49.443 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:10:49.443 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 23, 2024-04-17T06:10:49.443 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.130207837, 2024-04-17T06:10:49.443 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.005661210 2024-04-17T06:10:49.443 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.443 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:10:49.443 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4, 2024-04-17T06:10:49.443 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000275, 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000068 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 95616 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": [ 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.444 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 3113, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 4095, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 861409, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 1775678, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 7, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.179361712, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.117408053, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.112640012, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.001029130, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4095, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.076486752, 2024-04-17T06:10:49.445 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000018678 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4029, 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000615093, 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000152 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 113696, 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 13008 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": [ 2024-04-17T06:10:49.446 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.447 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.447 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.447 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-04-17T06:10:49.447 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-04-17T06:10:49.447 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.447 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.447 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.448 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": [ 2024-04-17T06:10:49.448 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.448 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.448 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.448 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-04-17T06:10:49.448 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-04-17T06:10:49.449 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2059, 2024-04-17T06:10:49.449 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.572115645, 2024-04-17T06:10:49.449 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000277860 2024-04-17T06:10:49.449 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.449 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.449 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.450 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.450 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": [ 2024-04-17T06:10:49.450 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.450 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.450 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.450 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-04-17T06:10:49.450 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-04-17T06:10:49.451 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-04-17T06:10:49.451 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000015348, 2024-04-17T06:10:49.454 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000007674 2024-04-17T06:10:49.454 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.454 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.454 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.455 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.455 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": [ 2024-04-17T06:10:49.455 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.455 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.455 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.455 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 1034, 2024-04-17T06:10:49.455 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 1034, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1034, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.693398527, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000670598 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 12, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 1012, 2024-04-17T06:10:49.456 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 1, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 1011, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 1011, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 1002, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 1003, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 2, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 1065, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 1062, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-04-17T06:10:49.457 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 997, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 2881, 2024-04-17T06:10:49.458 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 1, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 1000, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:10:49.459 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 1001, 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 1000, 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 3 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": [ 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-04-17T06:10:49.460 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-04-17T06:10:49.461 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.462 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_metrics": [ 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "cephfs", 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "5379" 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: "num_clients": 2 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_metrics-cephfs": [ 2024-04-17T06:10:49.463 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4943", 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "rank": "0" 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_hits": 43, 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_miss": 0, 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_read_latency": 0.000000000, 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_write_latency": 0.000000000, 2024-04-17T06:10:49.464 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_metadata_latency": 0.000000000, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_hits": 1, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_miss": 0, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_files": 0, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_inodes": 1, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "pinned_icaps": 1, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "total_inodes": 1, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_ops": 0, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_size": 0, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_ops": 0, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_size": 0 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.465 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5574", 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "rank": "0" 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_hits": 7354, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_miss": 20, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_read_latency": 0.000000000, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_write_latency": 0.000000000, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_metadata_latency": 0.000000000, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_hits": 816, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_miss": 0, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_files": 1, 2024-04-17T06:10:49.466 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_inodes": 815, 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "pinned_icaps": 815, 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "total_inodes": 815, 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_ops": 0, 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_size": 0, 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_ops": 813, 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_size": 2329 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": [ 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.467 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 2008, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 2008, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 3, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 3, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 2, 2024-04-17T06:10:49.468 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 8388020, 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1004, 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1.341506949, 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001336162 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": [ 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.469 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 1014, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 1014, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 0, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 13, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 13, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 0, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 1012, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 1012, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 0, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 1003, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 1003, 2024-04-17T06:10:49.470 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 55412, 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 231784 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": [ 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 1035, 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 1034, 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 1002, 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-04-17T06:10:49.471 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1000, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.686096504, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000686096 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 30, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.003793960, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000126465 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-04-17T06:10:49.472 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.473 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.474 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.475 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.003037913, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003037913 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.476 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000203295, 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000101647 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.477 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.478 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-04-17T06:10:49.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000266855, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000266855 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-04-17T06:10:49.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.481 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.482 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": [ 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 2, 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 2, 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 2, 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 1006, 2024-04-17T06:10:49.483 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 503, 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 11, 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": [ 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:10:49.484 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-04-17T06:10:49.485 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:10:49.485 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:10:49.485 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:10:49.485 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:10:49.485 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:10:49.485 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:10:49.486 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:10:49.486 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:10:49.486 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:10:49.486 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:10:49.486 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:10:49.486 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:10:49.486 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:10:49.486 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:10:49.487 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 256633, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 276, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 1760, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 20, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:10:49.488 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 5424, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 76, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 630796, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 6793, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.489 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": [ 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 1047, 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 4264357, 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 37, 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 1047, 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1047, 2024-04-17T06:10:49.490 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 2.143955505, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002047712 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1047, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1075 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 1047, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 0, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 1047, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 0, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 11, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 0, 2024-04-17T06:10:49.491 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 1004, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 8, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 6, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 22, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-04-17T06:10:49.492 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-04-17T06:10:49.493 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 38, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 1, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 8, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 8, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-04-17T06:10:49.494 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 23, 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 1 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": [ 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:10:49.495 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": [ 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": [ 2024-04-17T06:10:49.496 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.497 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 4178, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 796501, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 4178, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:10:49.498 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4178, 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 796501, 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": [ 2024-04-17T06:10:49.499 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:49.500 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:10:49.501 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 1047, 2024-04-17T06:10:49.501 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 4264357, 2024-04-17T06:10:49.501 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1047, 2024-04-17T06:10:49.501 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 4264357, 2024-04-17T06:10:49.501 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:49.502 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.502 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.502 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.502 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.502 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.502 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.503 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.503 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": [ 2024-04-17T06:10:49.503 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.503 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.503 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.503 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:49.503 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-04-17T06:10:49.504 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:49.504 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:10:49.504 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:10:49.504 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:49.504 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:10:49.504 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 1047, 2024-04-17T06:10:49.505 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 1047, 2024-04-17T06:10:49.505 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1047, 2024-04-17T06:10:49.505 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1047, 2024-04-17T06:10:49.505 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:49.505 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.505 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.506 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.506 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.506 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.506 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.506 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.506 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": [ 2024-04-17T06:10:49.507 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.507 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.507 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.507 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:49.507 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-04-17T06:10:49.507 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:49.508 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:10:49.508 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:10:49.508 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:49.508 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:10:49.508 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:10:49.508 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:10:49.509 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-04-17T06:10:49.509 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-04-17T06:10:49.509 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:49.509 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.509 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.509 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.510 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.510 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.510 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.510 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:10:49.510 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x56392f628500": [ 2024-04-17T06:10:49.510 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:10:49.511 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:10:49.511 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:10:49.511 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:10:49.511 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-04-17T06:10:49.511 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:10:49.511 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 2008, 2024-04-17T06:10:49.512 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 4193716, 2024-04-17T06:10:49.512 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:10:49.512 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 2008, 2024-04-17T06:10:49.512 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:10:49.512 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:10:49.512 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1004, 2024-04-17T06:10:49.513 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 4193716, 2024-04-17T06:10:49.513 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:10:49.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:10:49.513 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:10:49.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:10:49.513 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.514 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.514 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:10:49.514 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-17T06:10:49.514 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-17T06:10:49.516 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 -- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7feed006c8f0 msgr2=0x7feed006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7feed006c8f0 0x7feed006edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feee0006fd0 tx=0x7feee0008040 comp rx=0 tx=0).stop 2024-04-17T06:10:49.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 -- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 msgr2=0x7feef014d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:10:49.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef014d470 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feed800e810 tx=0x7feed800ecd0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 -- 172.21.15.135:0/3924583566 shutdown_connections 2024-04-17T06:10:49.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef01601f0 0x7feef0071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.517 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7feed006c8f0 0x7feed006edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef00d0300 0x7feef014d470 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 --2- 172.21.15.135:0/3924583566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef00cd2b0 0x7feef00b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:10:49.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 -- 172.21.15.135:0/3924583566 >> 172.21.15.135:0/3924583566 conn(0x7feef00b4190 msgr2=0x7feef00b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:10:49.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 -- 172.21.15.135:0/3924583566 shutdown_connections 2024-04-17T06:10:49.518 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:10:49.430+0000 7feef54ba640 1 -- 172.21.15.135:0/3924583566 wait complete. 2024-04-17T06:10:49.519 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:49.520 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_0", '"'"'r'"'"') as f: 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:49.537 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:49.538 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:49.538 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:49.538 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:49.538 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:49.538 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:49.538 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:49.538 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:49.579 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_0 2024-04-17T06:10:49.598 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_0 2024-04-17T06:10:49.598 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:49.598 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511627777 Links: 1 2024-04-17T06:10:49.599 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:49.599 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:49.599 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.227988364 +0000 2024-04-17T06:10:49.599 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.235988198 +0000 2024-04-17T06:10:49.599 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.235988198 +0000 2024-04-17T06:10:49.599 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.227988364 +0000 2024-04-17T06:10:49.599 INFO:tasks.cephfs.mount:File test_dir/test_file_0 became visible from 1 after 0s 2024-04-17T06:10:49.599 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:49.600 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:49.646 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:49.655 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:49.655 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:49.655 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_1", '"'"'r'"'"') as f: 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:49.656 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:49.687 DEBUG:teuthology.orchestra.run.smithi135:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:10:49.690 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-17T06:10:49.698 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_1 2024-04-17T06:10:49.717 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_1 2024-04-17T06:10:49.717 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:49.717 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628278 Links: 1 2024-04-17T06:10:49.717 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:49.717 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:49.717 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.239988115 +0000 2024-04-17T06:10:49.718 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.240988094 +0000 2024-04-17T06:10:49.718 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.240988094 +0000 2024-04-17T06:10:49.718 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.239988115 +0000 2024-04-17T06:10:49.718 INFO:tasks.cephfs.mount:File test_dir/test_file_1 became visible from 1 after 0s 2024-04-17T06:10:49.718 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:49.718 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:49.765 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_2", '"'"'r'"'"') as f: 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:49.780 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:49.781 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:49.781 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:49.781 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:49.781 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:49.781 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:49.822 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_2 2024-04-17T06:10:49.840 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_2 2024-04-17T06:10:49.840 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:49.840 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628279 Links: 1 2024-04-17T06:10:49.840 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:49.840 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:49.840 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.243988032 +0000 2024-04-17T06:10:49.841 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.244988011 +0000 2024-04-17T06:10:49.841 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.244988011 +0000 2024-04-17T06:10:49.841 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.243988032 +0000 2024-04-17T06:10:49.841 INFO:tasks.cephfs.mount:File test_dir/test_file_2 became visible from 1 after 0s 2024-04-17T06:10:49.841 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:49.841 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:49.887 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:49.897 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:49.897 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:49.897 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:49.897 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_3", '"'"'r'"'"') as f: 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:49.898 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:49.940 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_3 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_3 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628280 Links: 1 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.247987949 +0000 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.248987928 +0000 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.248987928 +0000 2024-04-17T06:10:49.959 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.247987949 +0000 2024-04-17T06:10:49.959 INFO:tasks.cephfs.mount:File test_dir/test_file_3 became visible from 1 after 0s 2024-04-17T06:10:49.960 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:49.960 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.006 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.015 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_4", '"'"'r'"'"') as f: 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.016 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:50.057 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_4 2024-04-17T06:10:50.075 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_4 2024-04-17T06:10:50.076 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:50.076 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628281 Links: 1 2024-04-17T06:10:50.076 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:50.076 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:50.076 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.251987866 +0000 2024-04-17T06:10:50.076 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.252987845 +0000 2024-04-17T06:10:50.076 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.252987845 +0000 2024-04-17T06:10:50.076 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.251987866 +0000 2024-04-17T06:10:50.076 INFO:tasks.cephfs.mount:File test_dir/test_file_4 became visible from 1 after 0s 2024-04-17T06:10:50.077 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:50.077 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.123 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_5", '"'"'r'"'"') as f: 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.132 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.133 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.133 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.133 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.133 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.133 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.133 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.133 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:50.174 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_5 2024-04-17T06:10:50.192 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_5 2024-04-17T06:10:50.192 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:50.193 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628282 Links: 1 2024-04-17T06:10:50.193 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:50.193 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:50.193 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.255987783 +0000 2024-04-17T06:10:50.193 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.255987783 +0000 2024-04-17T06:10:50.193 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.255987783 +0000 2024-04-17T06:10:50.193 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.255987783 +0000 2024-04-17T06:10:50.193 INFO:tasks.cephfs.mount:File test_dir/test_file_5 became visible from 1 after 0s 2024-04-17T06:10:50.193 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:50.193 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.240 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_6", '"'"'r'"'"') as f: 2024-04-17T06:10:50.249 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.250 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.250 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.250 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.250 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.250 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.250 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.250 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.250 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:50.291 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_6 2024-04-17T06:10:50.310 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_6 2024-04-17T06:10:50.310 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:50.310 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628283 Links: 1 2024-04-17T06:10:50.310 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:50.310 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:50.310 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.259987700 +0000 2024-04-17T06:10:50.310 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.260987679 +0000 2024-04-17T06:10:50.311 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.260987679 +0000 2024-04-17T06:10:50.311 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.259987700 +0000 2024-04-17T06:10:50.311 INFO:tasks.cephfs.mount:File test_dir/test_file_6 became visible from 1 after 0s 2024-04-17T06:10:50.311 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:50.311 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.355 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.367 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.367 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.367 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.367 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.367 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_7", '"'"'r'"'"') as f: 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.368 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:50.411 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_7 2024-04-17T06:10:50.432 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_7 2024-04-17T06:10:50.432 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:50.432 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628284 Links: 1 2024-04-17T06:10:50.432 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:50.432 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:50.433 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.263987617 +0000 2024-04-17T06:10:50.433 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.263987617 +0000 2024-04-17T06:10:50.433 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.263987617 +0000 2024-04-17T06:10:50.433 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.263987617 +0000 2024-04-17T06:10:50.433 INFO:tasks.cephfs.mount:File test_dir/test_file_7 became visible from 1 after 0s 2024-04-17T06:10:50.433 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:50.433 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.478 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_8", '"'"'r'"'"') as f: 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.489 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.490 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.490 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.490 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.490 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.490 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.490 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:50.531 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_8 2024-04-17T06:10:50.549 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_8 2024-04-17T06:10:50.549 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:50.550 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628285 Links: 1 2024-04-17T06:10:50.550 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:50.550 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:50.550 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.266987555 +0000 2024-04-17T06:10:50.550 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.267987534 +0000 2024-04-17T06:10:50.550 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.267987534 +0000 2024-04-17T06:10:50.550 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.266987555 +0000 2024-04-17T06:10:50.550 INFO:tasks.cephfs.mount:File test_dir/test_file_8 became visible from 1 after 0s 2024-04-17T06:10:50.550 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:50.550 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.595 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.606 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.606 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.606 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.606 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.606 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.606 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_9", '"'"'r'"'"') as f: 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.607 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:50.649 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_9 2024-04-17T06:10:50.667 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_9 2024-04-17T06:10:50.667 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 1 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:50.668 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628286 Links: 1 2024-04-17T06:10:50.668 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:50.668 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:50.668 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.270987472 +0000 2024-04-17T06:10:50.668 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.271987451 +0000 2024-04-17T06:10:50.668 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.271987451 +0000 2024-04-17T06:10:50.668 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.270987472 +0000 2024-04-17T06:10:50.668 INFO:tasks.cephfs.mount:File test_dir/test_file_9 became visible from 1 after 0s 2024-04-17T06:10:50.668 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:50.669 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.713 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_10", '"'"'r'"'"') as f: 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.724 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.725 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.725 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.725 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.725 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.725 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.725 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.725 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:50.766 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_10 2024-04-17T06:10:50.785 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_10 2024-04-17T06:10:50.786 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:50.786 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628287 Links: 1 2024-04-17T06:10:50.786 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:50.786 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:50.786 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.274987389 +0000 2024-04-17T06:10:50.786 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.275987368 +0000 2024-04-17T06:10:50.786 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.275987368 +0000 2024-04-17T06:10:50.786 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.274987389 +0000 2024-04-17T06:10:50.786 INFO:tasks.cephfs.mount:File test_dir/test_file_10 became visible from 1 after 0s 2024-04-17T06:10:50.786 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:50.787 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.831 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_11", '"'"'r'"'"') as f: 2024-04-17T06:10:50.849 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.850 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.850 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.850 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.850 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.850 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.850 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.850 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.850 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:50.892 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_11 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_11 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628288 Links: 1 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.278987305 +0000 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.279987284 +0000 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.279987284 +0000 2024-04-17T06:10:50.911 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.278987305 +0000 2024-04-17T06:10:50.912 INFO:tasks.cephfs.mount:File test_dir/test_file_11 became visible from 1 after 0s 2024-04-17T06:10:50.912 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:50.912 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:50.955 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:50.967 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:50.967 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:50.967 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:50.967 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:50.967 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:50.967 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_12", '"'"'r'"'"') as f: 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:50.968 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.009 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_12 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_12 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628289 Links: 1 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.282987222 +0000 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.283987201 +0000 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.283987201 +0000 2024-04-17T06:10:51.029 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.282987222 +0000 2024-04-17T06:10:51.030 INFO:tasks.cephfs.mount:File test_dir/test_file_12 became visible from 1 after 0s 2024-04-17T06:10:51.030 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.030 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.074 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.086 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.086 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.086 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.086 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_13", '"'"'r'"'"') as f: 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.087 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.128 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_13 2024-04-17T06:10:51.147 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_13 2024-04-17T06:10:51.148 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.148 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628290 Links: 1 2024-04-17T06:10:51.148 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.148 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.148 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.287987119 +0000 2024-04-17T06:10:51.148 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.288987098 +0000 2024-04-17T06:10:51.148 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.288987098 +0000 2024-04-17T06:10:51.148 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.287987119 +0000 2024-04-17T06:10:51.148 INFO:tasks.cephfs.mount:File test_dir/test_file_13 became visible from 1 after 0s 2024-04-17T06:10:51.148 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.149 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.192 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.204 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.204 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.204 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.204 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.204 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.204 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_14", '"'"'r'"'"') as f: 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.205 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.246 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_14 2024-04-17T06:10:51.265 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_14 2024-04-17T06:10:51.266 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.266 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628291 Links: 1 2024-04-17T06:10:51.266 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.266 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.266 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.292987015 +0000 2024-04-17T06:10:51.266 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.293986994 +0000 2024-04-17T06:10:51.266 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.293986994 +0000 2024-04-17T06:10:51.266 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.292987015 +0000 2024-04-17T06:10:51.266 INFO:tasks.cephfs.mount:File test_dir/test_file_14 became visible from 1 after 0s 2024-04-17T06:10:51.267 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.267 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.310 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.322 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.322 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.322 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.322 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.322 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.322 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_15", '"'"'r'"'"') as f: 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.323 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.364 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_15 2024-04-17T06:10:51.383 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_15 2024-04-17T06:10:51.383 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.383 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628292 Links: 1 2024-04-17T06:10:51.383 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.383 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.383 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.296986932 +0000 2024-04-17T06:10:51.383 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.297986911 +0000 2024-04-17T06:10:51.384 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.297986911 +0000 2024-04-17T06:10:51.384 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.296986932 +0000 2024-04-17T06:10:51.384 INFO:tasks.cephfs.mount:File test_dir/test_file_15 became visible from 1 after 0s 2024-04-17T06:10:51.384 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.384 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.428 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_16", '"'"'r'"'"') as f: 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.440 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.441 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.441 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.441 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.441 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.441 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.441 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.482 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_16 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_16 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628293 Links: 1 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.301986828 +0000 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.302986807 +0000 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.302986807 +0000 2024-04-17T06:10:51.501 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.301986828 +0000 2024-04-17T06:10:51.501 INFO:tasks.cephfs.mount:File test_dir/test_file_16 became visible from 1 after 0s 2024-04-17T06:10:51.502 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.502 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.546 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.557 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.557 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.557 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.557 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.557 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.557 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.557 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.557 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_17", '"'"'r'"'"') as f: 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.558 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.599 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_17 2024-04-17T06:10:51.618 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_17 2024-04-17T06:10:51.619 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.619 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628294 Links: 1 2024-04-17T06:10:51.619 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.619 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.619 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.305986745 +0000 2024-04-17T06:10:51.619 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.306986724 +0000 2024-04-17T06:10:51.619 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.306986724 +0000 2024-04-17T06:10:51.619 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.305986745 +0000 2024-04-17T06:10:51.619 INFO:tasks.cephfs.mount:File test_dir/test_file_17 became visible from 1 after 0s 2024-04-17T06:10:51.619 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.619 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.647 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.664 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.664 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.664 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.664 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.664 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.664 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.664 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_18", '"'"'r'"'"') as f: 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.665 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.666 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.668 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_18 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_18 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628295 Links: 1 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.312986600 +0000 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.313986579 +0000 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.313986579 +0000 2024-04-17T06:10:51.687 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.312986600 +0000 2024-04-17T06:10:51.687 INFO:tasks.cephfs.mount:File test_dir/test_file_18 became visible from 1 after 0s 2024-04-17T06:10:51.688 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.688 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.731 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.743 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.743 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.743 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.743 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.743 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.743 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_19", '"'"'r'"'"') as f: 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.744 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.768 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.769 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_19 2024-04-17T06:10:51.789 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_19 2024-04-17T06:10:51.789 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.790 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628296 Links: 1 2024-04-17T06:10:51.790 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.790 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.790 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.319986454 +0000 2024-04-17T06:10:51.790 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.320986433 +0000 2024-04-17T06:10:51.790 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.320986433 +0000 2024-04-17T06:10:51.790 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.319986454 +0000 2024-04-17T06:10:51.790 INFO:tasks.cephfs.mount:File test_dir/test_file_19 became visible from 1 after 0s 2024-04-17T06:10:51.790 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.791 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.835 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.846 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.846 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.846 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.846 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.846 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_20", '"'"'r'"'"') as f: 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.847 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:51.888 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_20 2024-04-17T06:10:51.890 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_20 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628297 Links: 1 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.323986371 +0000 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.324986350 +0000 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.324986350 +0000 2024-04-17T06:10:51.907 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.323986371 +0000 2024-04-17T06:10:51.908 INFO:tasks.cephfs.mount:File test_dir/test_file_20 became visible from 1 after 0s 2024-04-17T06:10:51.908 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:51.908 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:51.952 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_21", '"'"'r'"'"') as f: 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:51.963 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:51.964 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:51.964 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:51.964 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:51.964 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:51.964 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.005 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_21 2024-04-17T06:10:52.008 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.027 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_21 2024-04-17T06:10:52.027 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.027 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628298 Links: 1 2024-04-17T06:10:52.027 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.027 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.027 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.327986288 +0000 2024-04-17T06:10:52.027 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.327986288 +0000 2024-04-17T06:10:52.027 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.327986288 +0000 2024-04-17T06:10:52.028 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.327986288 +0000 2024-04-17T06:10:52.028 INFO:tasks.cephfs.mount:File test_dir/test_file_21 became visible from 1 after 0s 2024-04-17T06:10:52.028 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.028 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:52.070 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.083 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:52.083 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:52.083 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:52.083 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:52.083 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:52.083 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.083 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_22", '"'"'r'"'"') as f: 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:52.084 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.125 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_22 2024-04-17T06:10:52.126 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_22 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628299 Links: 1 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.330986226 +0000 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.331986205 +0000 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.331986205 +0000 2024-04-17T06:10:52.145 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.330986226 +0000 2024-04-17T06:10:52.145 INFO:tasks.cephfs.mount:File test_dir/test_file_22 became visible from 1 after 0s 2024-04-17T06:10:52.145 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.146 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:52.189 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_23", '"'"'r'"'"') as f: 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:52.201 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:52.202 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:52.202 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:52.202 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.242 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.244 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_23 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_23 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628300 Links: 1 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.334986143 +0000 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.335986122 +0000 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.335986122 +0000 2024-04-17T06:10:52.263 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.334986143 +0000 2024-04-17T06:10:52.263 INFO:tasks.cephfs.mount:File test_dir/test_file_23 became visible from 1 after 0s 2024-04-17T06:10:52.263 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.263 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:52.308 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.319 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:52.319 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:52.319 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:52.319 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:52.319 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:52.319 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.319 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_24", '"'"'r'"'"') as f: 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:52.320 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.356 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.358 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_24 2024-04-17T06:10:52.376 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_24 2024-04-17T06:10:52.376 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.377 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628301 Links: 1 2024-04-17T06:10:52.377 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.377 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.377 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.338986060 +0000 2024-04-17T06:10:52.377 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.338986060 +0000 2024-04-17T06:10:52.377 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.338986060 +0000 2024-04-17T06:10:52.377 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.338986060 +0000 2024-04-17T06:10:52.377 INFO:tasks.cephfs.mount:File test_dir/test_file_24 became visible from 1 after 0s 2024-04-17T06:10:52.377 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.377 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:52.422 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_25", '"'"'r'"'"') as f: 2024-04-17T06:10:52.434 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:52.435 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:52.435 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:52.435 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:52.435 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:52.435 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:52.435 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:52.435 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:52.435 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.476 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_25 2024-04-17T06:10:52.477 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.495 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_25 2024-04-17T06:10:52.495 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.495 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628302 Links: 1 2024-04-17T06:10:52.496 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.496 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.496 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.341985998 +0000 2024-04-17T06:10:52.496 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.342985977 +0000 2024-04-17T06:10:52.496 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.342985977 +0000 2024-04-17T06:10:52.496 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.341985998 +0000 2024-04-17T06:10:52.496 INFO:tasks.cephfs.mount:File test_dir/test_file_25 became visible from 1 after 0s 2024-04-17T06:10:52.496 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.496 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:52.540 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_26", '"'"'r'"'"') as f: 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:52.552 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:52.553 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:52.553 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:52.553 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:52.553 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:52.553 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:52.553 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:52.553 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.595 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_26 2024-04-17T06:10:52.597 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_26 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628303 Links: 1 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.344985936 +0000 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.345985915 +0000 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.345985915 +0000 2024-04-17T06:10:52.614 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.344985936 +0000 2024-04-17T06:10:52.614 INFO:tasks.cephfs.mount:File test_dir/test_file_26 became visible from 1 after 0s 2024-04-17T06:10:52.615 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.615 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:52.658 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_27", '"'"'r'"'"') as f: 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:52.670 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:52.671 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:52.671 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:52.671 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:52.671 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:52.671 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:52.671 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:52.671 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.712 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_27 2024-04-17T06:10:52.715 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.731 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_27 2024-04-17T06:10:52.731 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.732 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628304 Links: 1 2024-04-17T06:10:52.732 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.732 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.732 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.349985832 +0000 2024-04-17T06:10:52.732 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.350985811 +0000 2024-04-17T06:10:52.732 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.350985811 +0000 2024-04-17T06:10:52.732 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.349985832 +0000 2024-04-17T06:10:52.732 INFO:tasks.cephfs.mount:File test_dir/test_file_27 became visible from 1 after 0s 2024-04-17T06:10:52.732 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.732 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:52.776 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.788 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_28", '"'"'r'"'"') as f: 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:52.789 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.831 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_28 2024-04-17T06:10:52.833 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_28 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628305 Links: 1 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.353985749 +0000 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.354985728 +0000 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.354985728 +0000 2024-04-17T06:10:52.851 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.353985749 +0000 2024-04-17T06:10:52.852 INFO:tasks.cephfs.mount:File test_dir/test_file_28 became visible from 1 after 0s 2024-04-17T06:10:52.852 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.852 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:52.894 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.907 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:52.907 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:52.907 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:52.907 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_29", '"'"'r'"'"') as f: 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:52.908 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:52.909 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:52.909 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:52.909 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:52.951 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_29 2024-04-17T06:10:52.957 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:52.970 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_29 2024-04-17T06:10:52.970 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:52.970 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628306 Links: 1 2024-04-17T06:10:52.971 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:52.971 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:52.971 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.357985665 +0000 2024-04-17T06:10:52.971 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.357985665 +0000 2024-04-17T06:10:52.971 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.357985665 +0000 2024-04-17T06:10:52.971 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.357985665 +0000 2024-04-17T06:10:52.971 INFO:tasks.cephfs.mount:File test_dir/test_file_29 became visible from 1 after 0s 2024-04-17T06:10:52.971 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:52.971 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.014 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.029 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.029 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.029 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.029 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_30", '"'"'r'"'"') as f: 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.030 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.069 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_30 2024-04-17T06:10:53.076 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_30 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628307 Links: 1 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.361985582 +0000 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.362985562 +0000 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.362985562 +0000 2024-04-17T06:10:53.089 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.361985582 +0000 2024-04-17T06:10:53.089 INFO:tasks.cephfs.mount:File test_dir/test_file_30 became visible from 1 after 0s 2024-04-17T06:10:53.090 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.090 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.134 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_31", '"'"'r'"'"') as f: 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.146 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.147 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.147 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.147 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.147 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.147 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.188 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_31 2024-04-17T06:10:53.194 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_31 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628308 Links: 1 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.367985458 +0000 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.368985437 +0000 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.368985437 +0000 2024-04-17T06:10:53.207 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.367985458 +0000 2024-04-17T06:10:53.207 INFO:tasks.cephfs.mount:File test_dir/test_file_31 became visible from 1 after 0s 2024-04-17T06:10:53.207 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.208 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.253 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_32", '"'"'r'"'"') as f: 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.263 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.264 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.264 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.264 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.264 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.264 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.264 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.264 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.305 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_32 2024-04-17T06:10:53.312 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.324 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_32 2024-04-17T06:10:53.324 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.324 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628309 Links: 1 2024-04-17T06:10:53.325 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.325 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.325 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.371985375 +0000 2024-04-17T06:10:53.325 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.372985354 +0000 2024-04-17T06:10:53.325 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.372985354 +0000 2024-04-17T06:10:53.325 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.371985375 +0000 2024-04-17T06:10:53.325 INFO:tasks.cephfs.mount:File test_dir/test_file_32 became visible from 1 after 0s 2024-04-17T06:10:53.325 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.325 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.369 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_33", '"'"'r'"'"') as f: 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.381 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.382 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.382 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.382 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.382 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.382 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.423 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_33 2024-04-17T06:10:53.430 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.442 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_33 2024-04-17T06:10:53.443 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.443 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628310 Links: 1 2024-04-17T06:10:53.443 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.443 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.443 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.375985292 +0000 2024-04-17T06:10:53.443 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.376985271 +0000 2024-04-17T06:10:53.443 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.376985271 +0000 2024-04-17T06:10:53.444 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.375985292 +0000 2024-04-17T06:10:53.444 INFO:tasks.cephfs.mount:File test_dir/test_file_33 became visible from 1 after 0s 2024-04-17T06:10:53.444 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.444 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.487 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.499 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.499 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.499 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.499 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_34", '"'"'r'"'"') as f: 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.500 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.542 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_34 2024-04-17T06:10:53.549 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.562 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_34 2024-04-17T06:10:53.562 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.562 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628311 Links: 1 2024-04-17T06:10:53.562 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.563 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.563 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.379985209 +0000 2024-04-17T06:10:53.563 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.379985209 +0000 2024-04-17T06:10:53.563 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.379985209 +0000 2024-04-17T06:10:53.563 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.379985209 +0000 2024-04-17T06:10:53.563 INFO:tasks.cephfs.mount:File test_dir/test_file_34 became visible from 1 after 0s 2024-04-17T06:10:53.563 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.563 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.607 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.619 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.619 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.619 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_35", '"'"'r'"'"') as f: 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.620 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.621 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.650 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.652 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_35 2024-04-17T06:10:53.666 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.671 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_35 2024-04-17T06:10:53.671 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.671 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628312 Links: 1 2024-04-17T06:10:53.671 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.671 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.671 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.383985126 +0000 2024-04-17T06:10:53.672 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.384985105 +0000 2024-04-17T06:10:53.672 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.384985105 +0000 2024-04-17T06:10:53.672 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.383985126 +0000 2024-04-17T06:10:53.672 INFO:tasks.cephfs.mount:File test_dir/test_file_35 became visible from 1 after 0s 2024-04-17T06:10:53.672 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.672 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.716 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.728 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.728 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.728 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.728 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.728 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.728 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.728 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.728 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_36", '"'"'r'"'"') as f: 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.729 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.733 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.735 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_36 2024-04-17T06:10:53.753 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_36 2024-04-17T06:10:53.753 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.754 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628313 Links: 1 2024-04-17T06:10:53.754 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.754 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.754 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.387985043 +0000 2024-04-17T06:10:53.754 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.388985022 +0000 2024-04-17T06:10:53.754 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.388985022 +0000 2024-04-17T06:10:53.754 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.387985043 +0000 2024-04-17T06:10:53.754 INFO:tasks.cephfs.mount:File test_dir/test_file_36 became visible from 1 after 0s 2024-04-17T06:10:53.754 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.754 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.769 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.785 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.785 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.785 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_37", '"'"'r'"'"') as f: 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.786 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.799 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.801 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_37 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_37 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628314 Links: 1 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.391984960 +0000 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.392984939 +0000 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.392984939 +0000 2024-04-17T06:10:53.821 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.391984960 +0000 2024-04-17T06:10:53.821 INFO:tasks.cephfs.mount:File test_dir/test_file_37 became visible from 1 after 0s 2024-04-17T06:10:53.822 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.822 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.837 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.852 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_38", '"'"'r'"'"') as f: 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.853 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.854 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.854 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.867 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.869 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_38 2024-04-17T06:10:53.888 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_38 2024-04-17T06:10:53.888 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.888 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628315 Links: 1 2024-04-17T06:10:53.888 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.888 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.888 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.394984898 +0000 2024-04-17T06:10:53.888 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.395984877 +0000 2024-04-17T06:10:53.889 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.395984877 +0000 2024-04-17T06:10:53.889 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.394984898 +0000 2024-04-17T06:10:53.889 INFO:tasks.cephfs.mount:File test_dir/test_file_38 became visible from 1 after 0s 2024-04-17T06:10:53.889 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.889 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:53.892 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.907 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:53.907 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:53.907 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:53.907 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:53.907 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:53.907 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_39", '"'"'r'"'"') as f: 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:53.908 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:53.934 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.936 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_39 2024-04-17T06:10:53.953 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_39 2024-04-17T06:10:53.955 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:53.955 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628316 Links: 1 2024-04-17T06:10:53.956 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:53.956 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:53.956 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.398984814 +0000 2024-04-17T06:10:53.956 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.399984794 +0000 2024-04-17T06:10:53.956 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.399984794 +0000 2024-04-17T06:10:53.956 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.398984814 +0000 2024-04-17T06:10:53.956 INFO:tasks.cephfs.mount:File test_dir/test_file_39 became visible from 1 after 0s 2024-04-17T06:10:53.956 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:53.956 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.000 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.010 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.012 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.012 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.012 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.012 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.012 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.012 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_40", '"'"'r'"'"') as f: 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.013 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.014 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:54.054 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_40 2024-04-17T06:10:54.071 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_40 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628317 Links: 1 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.402984731 +0000 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.403984711 +0000 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.403984711 +0000 2024-04-17T06:10:54.074 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.402984731 +0000 2024-04-17T06:10:54.075 INFO:tasks.cephfs.mount:File test_dir/test_file_40 became visible from 1 after 0s 2024-04-17T06:10:54.075 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:54.075 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.119 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.127 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.130 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_41", '"'"'r'"'"') as f: 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.131 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.132 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.132 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.132 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:54.174 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_41 2024-04-17T06:10:54.190 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.193 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_41 2024-04-17T06:10:54.194 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:54.194 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628318 Links: 1 2024-04-17T06:10:54.194 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:54.194 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:54.194 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.406984649 +0000 2024-04-17T06:10:54.194 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.407984628 +0000 2024-04-17T06:10:54.194 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.407984628 +0000 2024-04-17T06:10:54.194 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.406984649 +0000 2024-04-17T06:10:54.194 INFO:tasks.cephfs.mount:File test_dir/test_file_41 became visible from 1 after 0s 2024-04-17T06:10:54.194 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:54.195 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.238 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.243 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.250 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_42", '"'"'r'"'"') as f: 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:54.292 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_42 2024-04-17T06:10:54.309 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.311 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_42 2024-04-17T06:10:54.311 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:54.311 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628319 Links: 1 2024-04-17T06:10:54.311 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:54.311 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:54.311 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.410984566 +0000 2024-04-17T06:10:54.312 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.411984545 +0000 2024-04-17T06:10:54.312 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.411984545 +0000 2024-04-17T06:10:54.312 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.410984566 +0000 2024-04-17T06:10:54.312 INFO:tasks.cephfs.mount:File test_dir/test_file_42 became visible from 1 after 0s 2024-04-17T06:10:54.312 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:54.312 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.356 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.357 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_43", '"'"'r'"'"') as f: 2024-04-17T06:10:54.368 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.369 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.369 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.369 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.369 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.369 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.369 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.369 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.369 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:54.411 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_43 2024-04-17T06:10:54.423 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.429 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_43 2024-04-17T06:10:54.429 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:54.430 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628320 Links: 1 2024-04-17T06:10:54.430 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:54.430 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:54.430 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.414984483 +0000 2024-04-17T06:10:54.430 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.415984462 +0000 2024-04-17T06:10:54.430 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.415984462 +0000 2024-04-17T06:10:54.430 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.414984483 +0000 2024-04-17T06:10:54.430 INFO:tasks.cephfs.mount:File test_dir/test_file_43 became visible from 1 after 0s 2024-04-17T06:10:54.430 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:54.430 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.474 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.477 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.486 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_44", '"'"'r'"'"') as f: 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.487 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:54.528 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_44 2024-04-17T06:10:54.541 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.547 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_44 2024-04-17T06:10:54.547 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:54.548 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628321 Links: 1 2024-04-17T06:10:54.548 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:54.548 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:54.548 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.418984400 +0000 2024-04-17T06:10:54.548 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.419984379 +0000 2024-04-17T06:10:54.548 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.419984379 +0000 2024-04-17T06:10:54.548 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.418984400 +0000 2024-04-17T06:10:54.548 INFO:tasks.cephfs.mount:File test_dir/test_file_44 became visible from 1 after 0s 2024-04-17T06:10:54.548 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:54.548 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.592 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.598 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.604 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.604 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.604 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.604 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.604 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.604 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.604 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.604 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_45", '"'"'r'"'"') as f: 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.605 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:54.646 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_45 2024-04-17T06:10:54.660 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_45 2024-04-17T06:10:54.665 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:54.666 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628322 Links: 1 2024-04-17T06:10:54.666 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:54.666 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:54.666 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.422984317 +0000 2024-04-17T06:10:54.666 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.423984296 +0000 2024-04-17T06:10:54.666 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.423984296 +0000 2024-04-17T06:10:54.666 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.422984317 +0000 2024-04-17T06:10:54.666 INFO:tasks.cephfs.mount:File test_dir/test_file_45 became visible from 1 after 0s 2024-04-17T06:10:54.666 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:54.666 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.711 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.716 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.722 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.722 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.722 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.722 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.722 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_46", '"'"'r'"'"') as f: 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.723 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:54.766 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_46 2024-04-17T06:10:54.778 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.785 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_46 2024-04-17T06:10:54.785 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:54.785 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628323 Links: 1 2024-04-17T06:10:54.785 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:54.785 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:54.785 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.428984192 +0000 2024-04-17T06:10:54.786 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.428984192 +0000 2024-04-17T06:10:54.786 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.428984192 +0000 2024-04-17T06:10:54.786 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.428984192 +0000 2024-04-17T06:10:54.786 INFO:tasks.cephfs.mount:File test_dir/test_file_46 became visible from 1 after 0s 2024-04-17T06:10:54.786 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:54.786 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.830 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.834 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.842 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_47", '"'"'r'"'"') as f: 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.843 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.844 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.844 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.844 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.844 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.844 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.844 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:54.885 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_47 2024-04-17T06:10:54.896 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.904 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_47 2024-04-17T06:10:54.904 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:54.904 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628324 Links: 1 2024-04-17T06:10:54.904 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:54.905 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:54.905 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.431984130 +0000 2024-04-17T06:10:54.905 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.432984109 +0000 2024-04-17T06:10:54.905 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.432984109 +0000 2024-04-17T06:10:54.905 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.431984130 +0000 2024-04-17T06:10:54.905 INFO:tasks.cephfs.mount:File test_dir/test_file_47 became visible from 1 after 0s 2024-04-17T06:10:54.905 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:54.905 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:54.950 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.959 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:54.961 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:54.961 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:54.961 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:54.961 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:54.961 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:54.961 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.961 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_48", '"'"'r'"'"') as f: 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:54.962 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.003 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_48 2024-04-17T06:10:55.016 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.023 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_48 2024-04-17T06:10:55.023 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.023 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628325 Links: 1 2024-04-17T06:10:55.023 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.023 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.023 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.435984047 +0000 2024-04-17T06:10:55.024 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.436984026 +0000 2024-04-17T06:10:55.024 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.436984026 +0000 2024-04-17T06:10:55.024 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.435984047 +0000 2024-04-17T06:10:55.024 INFO:tasks.cephfs.mount:File test_dir/test_file_48 became visible from 1 after 0s 2024-04-17T06:10:55.024 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.024 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.069 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.077 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.080 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.080 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.080 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.080 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.080 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_49", '"'"'r'"'"') as f: 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.081 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.123 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_49 2024-04-17T06:10:55.135 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_49 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628326 Links: 1 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.440983943 +0000 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.440983943 +0000 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.440983943 +0000 2024-04-17T06:10:55.142 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.440983943 +0000 2024-04-17T06:10:55.143 INFO:tasks.cephfs.mount:File test_dir/test_file_49 became visible from 1 after 0s 2024-04-17T06:10:55.143 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.143 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.188 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.196 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.198 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.198 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.198 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_50", '"'"'r'"'"') as f: 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.199 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.200 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.200 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.200 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.200 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.200 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.241 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_50 2024-04-17T06:10:55.254 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.260 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_50 2024-04-17T06:10:55.260 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.260 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628327 Links: 1 2024-04-17T06:10:55.260 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.261 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.261 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.443983881 +0000 2024-04-17T06:10:55.261 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.444983860 +0000 2024-04-17T06:10:55.261 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.444983860 +0000 2024-04-17T06:10:55.261 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.443983881 +0000 2024-04-17T06:10:55.261 INFO:tasks.cephfs.mount:File test_dir/test_file_50 became visible from 1 after 0s 2024-04-17T06:10:55.261 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.261 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.305 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.314 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.317 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.317 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.317 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.317 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.317 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.317 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_51", '"'"'r'"'"') as f: 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.318 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.319 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.360 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_51 2024-04-17T06:10:55.371 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.380 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_51 2024-04-17T06:10:55.380 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.380 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628328 Links: 1 2024-04-17T06:10:55.381 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.381 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.381 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.447983798 +0000 2024-04-17T06:10:55.381 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.448983777 +0000 2024-04-17T06:10:55.381 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.448983777 +0000 2024-04-17T06:10:55.381 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.447983798 +0000 2024-04-17T06:10:55.381 INFO:tasks.cephfs.mount:File test_dir/test_file_51 became visible from 1 after 0s 2024-04-17T06:10:55.381 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.381 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.424 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.432 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.437 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.437 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.437 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.437 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.437 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.437 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.437 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_52", '"'"'r'"'"') as f: 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.438 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.479 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_52 2024-04-17T06:10:55.489 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.498 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_52 2024-04-17T06:10:55.498 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.498 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628329 Links: 1 2024-04-17T06:10:55.499 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.499 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.499 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.451983715 +0000 2024-04-17T06:10:55.499 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.451983715 +0000 2024-04-17T06:10:55.499 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.451983715 +0000 2024-04-17T06:10:55.499 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.451983715 +0000 2024-04-17T06:10:55.499 INFO:tasks.cephfs.mount:File test_dir/test_file_52 became visible from 1 after 0s 2024-04-17T06:10:55.499 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.499 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.544 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.550 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_53", '"'"'r'"'"') as f: 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.556 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.557 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.557 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.557 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.557 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.557 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.557 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.557 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.598 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_53 2024-04-17T06:10:55.609 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_53 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628330 Links: 1 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.455983632 +0000 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.455983632 +0000 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.455983632 +0000 2024-04-17T06:10:55.618 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.455983632 +0000 2024-04-17T06:10:55.618 INFO:tasks.cephfs.mount:File test_dir/test_file_53 became visible from 1 after 0s 2024-04-17T06:10:55.619 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.619 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.652 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.663 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.667 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_54", '"'"'r'"'"') as f: 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.668 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.669 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.669 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.669 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.671 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_54 2024-04-17T06:10:55.690 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_54 2024-04-17T06:10:55.691 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.691 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628331 Links: 1 2024-04-17T06:10:55.691 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.691 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.691 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.458983569 +0000 2024-04-17T06:10:55.691 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.459983549 +0000 2024-04-17T06:10:55.691 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.459983549 +0000 2024-04-17T06:10:55.691 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.458983569 +0000 2024-04-17T06:10:55.691 INFO:tasks.cephfs.mount:File test_dir/test_file_54 became visible from 1 after 0s 2024-04-17T06:10:55.691 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.692 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.718 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.733 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.733 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.733 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.733 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.733 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.733 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.733 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_55", '"'"'r'"'"') as f: 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.734 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.735 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.735 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.737 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_55 2024-04-17T06:10:55.756 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_55 2024-04-17T06:10:55.756 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.756 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628332 Links: 1 2024-04-17T06:10:55.756 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.756 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.756 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.462983486 +0000 2024-04-17T06:10:55.757 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.463983466 +0000 2024-04-17T06:10:55.757 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.463983466 +0000 2024-04-17T06:10:55.757 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.462983486 +0000 2024-04-17T06:10:55.757 INFO:tasks.cephfs.mount:File test_dir/test_file_55 became visible from 1 after 0s 2024-04-17T06:10:55.757 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.757 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.772 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.787 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_56", '"'"'r'"'"') as f: 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.788 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.800 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.802 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_56 2024-04-17T06:10:55.803 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.821 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_56 2024-04-17T06:10:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.822 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628333 Links: 1 2024-04-17T06:10:55.822 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.822 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.822 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.466983403 +0000 2024-04-17T06:10:55.822 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.467983383 +0000 2024-04-17T06:10:55.822 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.467983383 +0000 2024-04-17T06:10:55.822 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.466983403 +0000 2024-04-17T06:10:55.822 INFO:tasks.cephfs.mount:File test_dir/test_file_56 became visible from 1 after 0s 2024-04-17T06:10:55.822 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.823 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.839 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.854 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_57", '"'"'r'"'"') as f: 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.855 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.856 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.856 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.856 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.867 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.868 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.869 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_57 2024-04-17T06:10:55.890 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_57 2024-04-17T06:10:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.891 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628334 Links: 1 2024-04-17T06:10:55.891 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.891 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.891 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.470983320 +0000 2024-04-17T06:10:55.891 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.470983320 +0000 2024-04-17T06:10:55.891 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.470983320 +0000 2024-04-17T06:10:55.891 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.470983320 +0000 2024-04-17T06:10:55.891 INFO:tasks.cephfs.mount:File test_dir/test_file_57 became visible from 1 after 0s 2024-04-17T06:10:55.891 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.891 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:55.893 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_58", '"'"'r'"'"') as f: 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:55.909 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:55.910 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:55.910 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:55.910 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:55.910 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:55.910 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:55.910 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:55.910 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:55.934 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.936 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_58 2024-04-17T06:10:55.937 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.956 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_58 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628335 Links: 1 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.474983237 +0000 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.474983237 +0000 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.474983237 +0000 2024-04-17T06:10:55.958 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.474983237 +0000 2024-04-17T06:10:55.959 INFO:tasks.cephfs.mount:File test_dir/test_file_58 became visible from 1 after 0s 2024-04-17T06:10:55.959 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:55.959 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.001 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.001 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.012 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.014 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.014 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.014 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.014 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.014 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.014 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.014 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_59", '"'"'r'"'"') as f: 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.015 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:56.056 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_59 2024-04-17T06:10:56.073 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.076 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_59 2024-04-17T06:10:56.076 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:56.077 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628336 Links: 1 2024-04-17T06:10:56.077 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:56.077 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:56.077 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.479983133 +0000 2024-04-17T06:10:56.077 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.479983133 +0000 2024-04-17T06:10:56.077 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.479983133 +0000 2024-04-17T06:10:56.077 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.479983133 +0000 2024-04-17T06:10:56.077 INFO:tasks.cephfs.mount:File test_dir/test_file_59 became visible from 1 after 0s 2024-04-17T06:10:56.077 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:56.077 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.121 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.121 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.129 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.133 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.133 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.133 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.133 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.133 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.133 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.133 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.133 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_60", '"'"'r'"'"') as f: 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.134 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:56.175 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_60 2024-04-17T06:10:56.193 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.195 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_60 2024-04-17T06:10:56.195 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:56.196 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628337 Links: 1 2024-04-17T06:10:56.196 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:56.196 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:56.196 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.483983050 +0000 2024-04-17T06:10:56.196 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.484983030 +0000 2024-04-17T06:10:56.196 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.484983030 +0000 2024-04-17T06:10:56.197 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.483983050 +0000 2024-04-17T06:10:56.197 INFO:tasks.cephfs.mount:File test_dir/test_file_60 became visible from 1 after 0s 2024-04-17T06:10:56.197 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:56.197 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.240 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.240 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.245 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.252 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.252 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.252 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.252 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_61", '"'"'r'"'"') as f: 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.253 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:56.295 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_61 2024-04-17T06:10:56.311 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_61 2024-04-17T06:10:56.315 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:56.315 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628338 Links: 1 2024-04-17T06:10:56.315 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:56.315 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:56.315 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.487982967 +0000 2024-04-17T06:10:56.315 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.487982967 +0000 2024-04-17T06:10:56.316 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.487982967 +0000 2024-04-17T06:10:56.316 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.487982967 +0000 2024-04-17T06:10:56.316 INFO:tasks.cephfs.mount:File test_dir/test_file_61 became visible from 1 after 0s 2024-04-17T06:10:56.316 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:56.316 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.359 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.359 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.360 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.372 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.372 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.372 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.372 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.372 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_62", '"'"'r'"'"') as f: 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.373 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:56.415 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_62 2024-04-17T06:10:56.425 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_62 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628339 Links: 1 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.490982905 +0000 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.491982884 +0000 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.491982884 +0000 2024-04-17T06:10:56.434 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.490982905 +0000 2024-04-17T06:10:56.435 INFO:tasks.cephfs.mount:File test_dir/test_file_62 became visible from 1 after 0s 2024-04-17T06:10:56.435 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:56.435 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.476 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.478 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.479 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_63", '"'"'r'"'"') as f: 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.490 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.491 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.491 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.491 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.491 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.491 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.491 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:56.532 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_63 2024-04-17T06:10:56.543 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.551 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_63 2024-04-17T06:10:56.552 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:56.552 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628340 Links: 1 2024-04-17T06:10:56.552 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:56.552 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:56.552 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.494982822 +0000 2024-04-17T06:10:56.552 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.495982802 +0000 2024-04-17T06:10:56.552 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.495982802 +0000 2024-04-17T06:10:56.552 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.494982822 +0000 2024-04-17T06:10:56.552 INFO:tasks.cephfs.mount:File test_dir/test_file_63 became visible from 1 after 0s 2024-04-17T06:10:56.552 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:56.552 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.595 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.596 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.600 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.608 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.608 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.608 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.608 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.608 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.608 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.608 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.608 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_64", '"'"'r'"'"') as f: 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.609 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:56.650 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_64 2024-04-17T06:10:56.662 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.670 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_64 2024-04-17T06:10:56.670 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:56.670 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628341 Links: 1 2024-04-17T06:10:56.671 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:56.671 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:56.671 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.498982739 +0000 2024-04-17T06:10:56.671 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.499982718 +0000 2024-04-17T06:10:56.671 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.499982718 +0000 2024-04-17T06:10:56.671 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.498982739 +0000 2024-04-17T06:10:56.671 INFO:tasks.cephfs.mount:File test_dir/test_file_64 became visible from 1 after 0s 2024-04-17T06:10:56.671 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:56.671 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.713 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.715 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.718 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.728 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.728 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.728 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.728 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.728 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.728 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_65", '"'"'r'"'"') as f: 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.729 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:56.771 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_65 2024-04-17T06:10:56.779 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.790 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_65 2024-04-17T06:10:56.790 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:56.790 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628342 Links: 1 2024-04-17T06:10:56.791 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:56.791 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:56.791 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.502982656 +0000 2024-04-17T06:10:56.791 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.502982656 +0000 2024-04-17T06:10:56.791 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.502982656 +0000 2024-04-17T06:10:56.791 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.502982656 +0000 2024-04-17T06:10:56.791 INFO:tasks.cephfs.mount:File test_dir/test_file_65 became visible from 1 after 0s 2024-04-17T06:10:56.791 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:56.791 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.832 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.835 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.836 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.847 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.847 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.847 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.847 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_66", '"'"'r'"'"') as f: 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.848 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:56.890 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_66 2024-04-17T06:10:56.898 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.910 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_66 2024-04-17T06:10:56.910 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:56.910 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628343 Links: 1 2024-04-17T06:10:56.910 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:56.910 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:56.910 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.505982594 +0000 2024-04-17T06:10:56.911 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.506982573 +0000 2024-04-17T06:10:56.911 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.506982573 +0000 2024-04-17T06:10:56.911 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.505982594 +0000 2024-04-17T06:10:56.911 INFO:tasks.cephfs.mount:File test_dir/test_file_66 became visible from 1 after 0s 2024-04-17T06:10:56.911 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:56.911 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:56.953 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.954 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.961 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:56.967 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:56.967 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:56.967 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:56.967 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:56.967 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:56.967 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_67", '"'"'r'"'"') as f: 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:56.968 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.009 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_67 2024-04-17T06:10:57.018 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_67 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628344 Links: 1 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.509982511 +0000 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.510982490 +0000 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.510982490 +0000 2024-04-17T06:10:57.029 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.509982511 +0000 2024-04-17T06:10:57.030 INFO:tasks.cephfs.mount:File test_dir/test_file_67 became visible from 1 after 0s 2024-04-17T06:10:57.030 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.030 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.072 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.074 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.079 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.085 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_68", '"'"'r'"'"') as f: 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.086 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.127 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_68 2024-04-17T06:10:57.137 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.147 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_68 2024-04-17T06:10:57.148 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.148 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628345 Links: 1 2024-04-17T06:10:57.148 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.148 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.148 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.513982428 +0000 2024-04-17T06:10:57.148 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.513982428 +0000 2024-04-17T06:10:57.148 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.513982428 +0000 2024-04-17T06:10:57.148 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.513982428 +0000 2024-04-17T06:10:57.149 INFO:tasks.cephfs.mount:File test_dir/test_file_68 became visible from 1 after 0s 2024-04-17T06:10:57.149 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.149 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.191 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.191 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.198 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.204 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_69", '"'"'r'"'"') as f: 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.205 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.246 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_69 2024-04-17T06:10:57.256 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.269 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_69 2024-04-17T06:10:57.269 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.269 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628346 Links: 1 2024-04-17T06:10:57.269 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.269 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.269 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.516982365 +0000 2024-04-17T06:10:57.270 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.517982344 +0000 2024-04-17T06:10:57.270 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.517982344 +0000 2024-04-17T06:10:57.270 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.516982365 +0000 2024-04-17T06:10:57.270 INFO:tasks.cephfs.mount:File test_dir/test_file_69 became visible from 1 after 0s 2024-04-17T06:10:57.270 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.270 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.310 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.323 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.323 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.323 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.323 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.323 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.323 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.323 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_70", '"'"'r'"'"') as f: 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.324 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.365 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_70 2024-04-17T06:10:57.373 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_70 2024-04-17T06:10:57.385 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.385 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628347 Links: 1 2024-04-17T06:10:57.385 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.385 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.385 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.520982282 +0000 2024-04-17T06:10:57.385 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.520982282 +0000 2024-04-17T06:10:57.385 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.520982282 +0000 2024-04-17T06:10:57.386 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.520982282 +0000 2024-04-17T06:10:57.386 INFO:tasks.cephfs.mount:File test_dir/test_file_70 became visible from 1 after 0s 2024-04-17T06:10:57.386 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.386 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.427 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.429 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.434 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.442 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_71", '"'"'r'"'"') as f: 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.443 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.485 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_71 2024-04-17T06:10:57.491 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.504 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_71 2024-04-17T06:10:57.504 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.504 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628348 Links: 1 2024-04-17T06:10:57.504 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.505 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.505 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.523982220 +0000 2024-04-17T06:10:57.505 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.524982199 +0000 2024-04-17T06:10:57.505 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.524982199 +0000 2024-04-17T06:10:57.505 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.523982220 +0000 2024-04-17T06:10:57.505 INFO:tasks.cephfs.mount:File test_dir/test_file_71 became visible from 1 after 0s 2024-04-17T06:10:57.505 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.505 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.546 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.549 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.551 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.561 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.561 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.561 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.561 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.561 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.561 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.561 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.561 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_72", '"'"'r'"'"') as f: 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.562 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.603 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_72 2024-04-17T06:10:57.610 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_72 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628349 Links: 1 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.527982137 +0000 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.527982137 +0000 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.527982137 +0000 2024-04-17T06:10:57.623 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.527982137 +0000 2024-04-17T06:10:57.624 INFO:tasks.cephfs.mount:File test_dir/test_file_72 became visible from 1 after 0s 2024-04-17T06:10:57.624 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.624 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.654 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.664 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.667 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.669 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.669 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.669 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.669 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.669 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.669 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.669 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.669 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_73", '"'"'r'"'"') as f: 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.670 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.671 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.672 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_73 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_73 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628350 Links: 1 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.530982075 +0000 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.531982054 +0000 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.531982054 +0000 2024-04-17T06:10:57.692 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.530982075 +0000 2024-04-17T06:10:57.692 INFO:tasks.cephfs.mount:File test_dir/test_file_73 became visible from 1 after 0s 2024-04-17T06:10:57.693 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.693 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.720 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_74", '"'"'r'"'"') as f: 2024-04-17T06:10:57.735 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.736 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.736 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.736 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.736 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.736 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.736 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.736 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.736 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.737 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.737 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.737 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.738 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_74 2024-04-17T06:10:57.759 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_74 2024-04-17T06:10:57.759 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.759 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628351 Links: 1 2024-04-17T06:10:57.759 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.759 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.759 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.534981992 +0000 2024-04-17T06:10:57.759 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.535981971 +0000 2024-04-17T06:10:57.760 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.535981971 +0000 2024-04-17T06:10:57.760 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.534981992 +0000 2024-04-17T06:10:57.760 INFO:tasks.cephfs.mount:File test_dir/test_file_74 became visible from 1 after 0s 2024-04-17T06:10:57.760 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.760 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.773 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.789 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.789 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.789 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.789 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_75", '"'"'r'"'"') as f: 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.790 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.791 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.803 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.803 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.805 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_75 2024-04-17T06:10:57.805 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.824 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_75 2024-04-17T06:10:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.825 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628352 Links: 1 2024-04-17T06:10:57.825 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.825 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.825 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.537981930 +0000 2024-04-17T06:10:57.825 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.538981909 +0000 2024-04-17T06:10:57.825 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.538981909 +0000 2024-04-17T06:10:57.825 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.537981930 +0000 2024-04-17T06:10:57.825 INFO:tasks.cephfs.mount:File test_dir/test_file_75 became visible from 1 after 0s 2024-04-17T06:10:57.825 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.825 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.841 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.857 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_76", '"'"'r'"'"') as f: 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.858 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.869 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.869 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.870 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_76 2024-04-17T06:10:57.871 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_76 2024-04-17T06:10:57.890 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.890 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628353 Links: 1 2024-04-17T06:10:57.890 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.890 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.890 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.541981847 +0000 2024-04-17T06:10:57.890 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.542981826 +0000 2024-04-17T06:10:57.890 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.542981826 +0000 2024-04-17T06:10:57.891 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.541981847 +0000 2024-04-17T06:10:57.891 INFO:tasks.cephfs.mount:File test_dir/test_file_76 became visible from 1 after 0s 2024-04-17T06:10:57.891 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.891 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.895 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_77", '"'"'r'"'"') as f: 2024-04-17T06:10:57.911 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.912 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.912 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.912 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.912 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.912 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.912 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.912 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.912 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:57.935 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.936 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.937 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_77 2024-04-17T06:10:57.938 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.957 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_77 2024-04-17T06:10:57.957 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:57.957 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628354 Links: 1 2024-04-17T06:10:57.957 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:57.957 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:57.957 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.545981764 +0000 2024-04-17T06:10:57.957 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.545981764 +0000 2024-04-17T06:10:57.958 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.545981764 +0000 2024-04-17T06:10:57.958 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.545981764 +0000 2024-04-17T06:10:57.958 INFO:tasks.cephfs.mount:File test_dir/test_file_77 became visible from 1 after 0s 2024-04-17T06:10:57.958 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:57.958 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:57.959 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_78", '"'"'r'"'"') as f: 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:57.976 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:57.977 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:57.977 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:57.977 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:57.977 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:57.977 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.002 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.004 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_78 2024-04-17T06:10:58.004 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.005 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.014 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.024 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_78 2024-04-17T06:10:58.024 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.024 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628355 Links: 1 2024-04-17T06:10:58.024 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.024 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.024 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.548981701 +0000 2024-04-17T06:10:58.024 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.549981681 +0000 2024-04-17T06:10:58.024 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.549981681 +0000 2024-04-17T06:10:58.025 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.548981701 +0000 2024-04-17T06:10:58.025 INFO:tasks.cephfs.mount:File test_dir/test_file_78 became visible from 1 after 0s 2024-04-17T06:10:58.025 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.025 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:58.069 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.075 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.081 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:58.081 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:58.081 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:58.081 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:58.081 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:58.081 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.081 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:58.081 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_79", '"'"'r'"'"') as f: 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:58.082 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.124 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_79 2024-04-17T06:10:58.124 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.125 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.131 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.143 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_79 2024-04-17T06:10:58.143 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.143 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628356 Links: 1 2024-04-17T06:10:58.143 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.143 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.144 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.552981618 +0000 2024-04-17T06:10:58.144 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.553981598 +0000 2024-04-17T06:10:58.144 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.553981598 +0000 2024-04-17T06:10:58.144 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.552981618 +0000 2024-04-17T06:10:58.144 INFO:tasks.cephfs.mount:File test_dir/test_file_79 became visible from 1 after 0s 2024-04-17T06:10:58.144 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.144 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:58.187 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.194 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.200 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:58.200 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:58.200 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:58.200 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:58.200 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:58.200 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.200 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:58.200 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_80", '"'"'r'"'"') as f: 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:58.201 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.242 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_80 2024-04-17T06:10:58.243 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.243 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.247 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.262 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_80 2024-04-17T06:10:58.262 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.262 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628357 Links: 1 2024-04-17T06:10:58.263 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.263 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.263 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.558981494 +0000 2024-04-17T06:10:58.263 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.559981473 +0000 2024-04-17T06:10:58.263 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.559981473 +0000 2024-04-17T06:10:58.263 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.558981494 +0000 2024-04-17T06:10:58.263 INFO:tasks.cephfs.mount:File test_dir/test_file_80 became visible from 1 after 0s 2024-04-17T06:10:58.263 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.263 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.313 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.319 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_81", '"'"'r'"'"') as f: 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:58.320 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:58.321 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:58.321 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:58.321 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.360 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.362 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_81 2024-04-17T06:10:58.362 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.363 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_81 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628358 Links: 1 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.562981411 +0000 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.563981390 +0000 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.563981390 +0000 2024-04-17T06:10:58.382 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.562981411 +0000 2024-04-17T06:10:58.382 INFO:tasks.cephfs.mount:File test_dir/test_file_81 became visible from 1 after 0s 2024-04-17T06:10:58.383 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.383 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:58.426 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.427 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.438 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:58.438 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:58.438 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_82", '"'"'r'"'"') as f: 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:58.439 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.478 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.480 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_82 2024-04-17T06:10:58.481 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.482 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.508 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_82 2024-04-17T06:10:58.508 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.508 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628359 Links: 1 2024-04-17T06:10:58.508 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.508 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.509 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.566981328 +0000 2024-04-17T06:10:58.509 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.566981328 +0000 2024-04-17T06:10:58.509 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.566981328 +0000 2024-04-17T06:10:58.509 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.566981328 +0000 2024-04-17T06:10:58.509 INFO:tasks.cephfs.mount:File test_dir/test_file_82 became visible from 1 after 0s 2024-04-17T06:10:58.509 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.509 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:58.544 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.546 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.560 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_83", '"'"'r'"'"') as f: 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:58.561 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.597 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.598 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_83 2024-04-17T06:10:58.599 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.602 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_83 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628360 Links: 1 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.569981265 +0000 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.570981245 +0000 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.570981245 +0000 2024-04-17T06:10:58.618 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.569981265 +0000 2024-04-17T06:10:58.618 INFO:tasks.cephfs.mount:File test_dir/test_file_83 became visible from 1 after 0s 2024-04-17T06:10:58.619 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.619 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:58.663 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.663 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.674 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:58.674 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:58.674 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:58.674 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:58.674 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_84", '"'"'r'"'"') as f: 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:58.675 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.715 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.715 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.717 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_84 2024-04-17T06:10:58.720 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.740 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_84 2024-04-17T06:10:58.740 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.741 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628361 Links: 1 2024-04-17T06:10:58.741 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.741 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.741 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.573981183 +0000 2024-04-17T06:10:58.741 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.573981183 +0000 2024-04-17T06:10:58.741 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.573981183 +0000 2024-04-17T06:10:58.741 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.573981183 +0000 2024-04-17T06:10:58.741 INFO:tasks.cephfs.mount:File test_dir/test_file_84 became visible from 1 after 0s 2024-04-17T06:10:58.741 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.741 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:58.782 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.782 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.797 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_85", '"'"'r'"'"') as f: 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:58.798 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:58.799 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:58.799 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:58.799 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.834 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.836 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_85 2024-04-17T06:10:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_85 2024-04-17T06:10:58.855 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.855 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628362 Links: 1 2024-04-17T06:10:58.855 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.855 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.855 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.576981120 +0000 2024-04-17T06:10:58.855 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.577981099 +0000 2024-04-17T06:10:58.855 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.577981099 +0000 2024-04-17T06:10:58.856 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.576981120 +0000 2024-04-17T06:10:58.856 INFO:tasks.cephfs.mount:File test_dir/test_file_85 became visible from 1 after 0s 2024-04-17T06:10:58.856 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.856 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:58.899 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.900 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.911 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:58.911 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:58.911 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:58.911 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_86", '"'"'r'"'"') as f: 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:58.912 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:58.954 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_86 2024-04-17T06:10:58.955 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.957 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.964 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_86 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628363 Links: 1 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.580981037 +0000 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.581981016 +0000 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.581981016 +0000 2024-04-17T06:10:58.974 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.580981037 +0000 2024-04-17T06:10:58.974 INFO:tasks.cephfs.mount:File test_dir/test_file_86 became visible from 1 after 0s 2024-04-17T06:10:58.975 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:58.975 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.019 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.020 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_87", '"'"'r'"'"') as f: 2024-04-17T06:10:59.030 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.031 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.031 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.031 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.031 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.031 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.031 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.031 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.031 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.074 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.075 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_87 2024-04-17T06:10:59.076 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.081 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.094 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_87 2024-04-17T06:10:59.095 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.095 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628364 Links: 1 2024-04-17T06:10:59.095 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.095 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.095 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.584980954 +0000 2024-04-17T06:10:59.095 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.584980954 +0000 2024-04-17T06:10:59.095 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.584980954 +0000 2024-04-17T06:10:59.095 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.584980954 +0000 2024-04-17T06:10:59.095 INFO:tasks.cephfs.mount:File test_dir/test_file_87 became visible from 1 after 0s 2024-04-17T06:10:59.095 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.096 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.140 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.151 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.151 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.151 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.151 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.151 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.151 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_88", '"'"'r'"'"') as f: 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.152 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.192 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.192 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.193 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_88 2024-04-17T06:10:59.200 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_88 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628365 Links: 1 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.588980871 +0000 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.588980871 +0000 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.588980871 +0000 2024-04-17T06:10:59.213 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.588980871 +0000 2024-04-17T06:10:59.213 INFO:tasks.cephfs.mount:File test_dir/test_file_88 became visible from 1 after 0s 2024-04-17T06:10:59.214 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.214 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.257 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.258 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_89", '"'"'r'"'"') as f: 2024-04-17T06:10:59.269 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.270 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.270 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.270 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.270 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.270 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.270 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.270 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.270 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.309 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.311 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_89 2024-04-17T06:10:59.312 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.318 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_89 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628366 Links: 1 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.591980809 +0000 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.592980788 +0000 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.592980788 +0000 2024-04-17T06:10:59.332 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.591980809 +0000 2024-04-17T06:10:59.332 INFO:tasks.cephfs.mount:File test_dir/test_file_89 became visible from 1 after 0s 2024-04-17T06:10:59.333 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.333 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.375 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.376 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.388 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.388 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.388 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.388 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.388 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.388 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_90", '"'"'r'"'"') as f: 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.389 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.429 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.431 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_90 2024-04-17T06:10:59.432 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.436 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.455 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_90 2024-04-17T06:10:59.455 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.455 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628367 Links: 1 2024-04-17T06:10:59.455 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.455 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.455 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.595980726 +0000 2024-04-17T06:10:59.456 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.596980705 +0000 2024-04-17T06:10:59.456 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.596980705 +0000 2024-04-17T06:10:59.456 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.595980726 +0000 2024-04-17T06:10:59.456 INFO:tasks.cephfs.mount:File test_dir/test_file_90 became visible from 1 after 0s 2024-04-17T06:10:59.456 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.456 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.494 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.498 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.509 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.509 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.509 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_91", '"'"'r'"'"') as f: 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.510 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.548 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.550 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_91 2024-04-17T06:10:59.550 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.553 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.569 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_91 2024-04-17T06:10:59.569 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.569 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628368 Links: 1 2024-04-17T06:10:59.569 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.569 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.570 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.599980643 +0000 2024-04-17T06:10:59.570 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.600980622 +0000 2024-04-17T06:10:59.570 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.600980622 +0000 2024-04-17T06:10:59.570 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.599980643 +0000 2024-04-17T06:10:59.570 INFO:tasks.cephfs.mount:File test_dir/test_file_91 became visible from 1 after 0s 2024-04-17T06:10:59.570 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.570 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.612 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.613 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.625 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_92", '"'"'r'"'"') as f: 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.626 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.656 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.657 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_92 2024-04-17T06:10:59.666 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.669 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.672 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.676 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_92 2024-04-17T06:10:59.676 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.677 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628369 Links: 1 2024-04-17T06:10:59.677 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.677 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.677 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.603980560 +0000 2024-04-17T06:10:59.677 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.603980560 +0000 2024-04-17T06:10:59.677 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.603980560 +0000 2024-04-17T06:10:59.677 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.603980560 +0000 2024-04-17T06:10:59.677 INFO:tasks.cephfs.mount:File test_dir/test_file_92 became visible from 1 after 0s 2024-04-17T06:10:59.677 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.677 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.721 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.722 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.734 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.734 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.734 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_93", '"'"'r'"'"') as f: 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.735 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.738 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.739 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.739 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.740 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_93 2024-04-17T06:10:59.760 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_93 2024-04-17T06:10:59.760 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.760 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628370 Links: 1 2024-04-17T06:10:59.760 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.760 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.760 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.607980477 +0000 2024-04-17T06:10:59.760 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.607980477 +0000 2024-04-17T06:10:59.761 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.607980477 +0000 2024-04-17T06:10:59.761 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.607980477 +0000 2024-04-17T06:10:59.761 INFO:tasks.cephfs.mount:File test_dir/test_file_93 became visible from 1 after 0s 2024-04-17T06:10:59.761 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.761 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.776 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.793 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.793 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.793 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.793 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.793 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.793 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.793 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_94", '"'"'r'"'"') as f: 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.794 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.805 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.805 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.806 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.806 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_94 2024-04-17T06:10:59.808 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.825 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_94 2024-04-17T06:10:59.826 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.826 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628371 Links: 1 2024-04-17T06:10:59.826 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.826 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.826 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.610980415 +0000 2024-04-17T06:10:59.826 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.611980394 +0000 2024-04-17T06:10:59.826 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.611980394 +0000 2024-04-17T06:10:59.826 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.610980415 +0000 2024-04-17T06:10:59.826 INFO:tasks.cephfs.mount:File test_dir/test_file_94 became visible from 1 after 0s 2024-04-17T06:10:59.827 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.827 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.843 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.858 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.858 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_95", '"'"'r'"'"') as f: 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.859 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.870 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.871 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.872 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_95 2024-04-17T06:10:59.873 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.873 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.892 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_95 2024-04-17T06:10:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.893 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628372 Links: 1 2024-04-17T06:10:59.893 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.893 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.893 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.614980332 +0000 2024-04-17T06:10:59.893 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.615980311 +0000 2024-04-17T06:10:59.893 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.615980311 +0000 2024-04-17T06:10:59.893 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.614980332 +0000 2024-04-17T06:10:59.893 INFO:tasks.cephfs.mount:File test_dir/test_file_95 became visible from 1 after 0s 2024-04-17T06:10:59.893 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.894 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.898 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_96", '"'"'r'"'"') as f: 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:10:59.914 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:10:59.915 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:10:59.915 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:10:59.915 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:10:59.915 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:10:59.915 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:10:59.937 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.938 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.938 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.940 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_96 2024-04-17T06:10:59.941 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:10:59.961 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_96 2024-04-17T06:10:59.961 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:10:59.961 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628373 Links: 1 2024-04-17T06:10:59.961 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:10:59.961 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:10:59.961 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.619980228 +0000 2024-04-17T06:10:59.962 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.620980207 +0000 2024-04-17T06:10:59.962 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.620980207 +0000 2024-04-17T06:10:59.962 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.619980228 +0000 2024-04-17T06:10:59.962 INFO:tasks.cephfs.mount:File test_dir/test_file_96 became visible from 1 after 0s 2024-04-17T06:10:59.962 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:10:59.962 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:10:59.963 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.004 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.005 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.005 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.007 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:11:00.017 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_97", '"'"'r'"'"') as f: 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:11:00.018 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:11:00.018 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.059 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_97 2024-04-17T06:11:00.071 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.078 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.079 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_97 2024-04-17T06:11:00.079 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:11:00.079 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628374 Links: 1 2024-04-17T06:11:00.079 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:11:00.079 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:11:00.079 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.625980103 +0000 2024-04-17T06:11:00.079 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.625980103 +0000 2024-04-17T06:11:00.080 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.625980103 +0000 2024-04-17T06:11:00.080 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.625980103 +0000 2024-04-17T06:11:00.080 INFO:tasks.cephfs.mount:File test_dir/test_file_97 became visible from 1 after 0s 2024-04-17T06:11:00.080 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:11:00.080 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:11:00.125 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.125 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.125 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.133 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.136 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:11:00.136 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:11:00.136 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_98", '"'"'r'"'"') as f: 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:11:00.137 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:11:00.179 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_98 2024-04-17T06:11:00.189 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.196 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.199 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_98 2024-04-17T06:11:00.199 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:11:00.199 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628375 Links: 1 2024-04-17T06:11:00.199 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:11:00.199 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:11:00.199 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.630979999 +0000 2024-04-17T06:11:00.199 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.631979979 +0000 2024-04-17T06:11:00.200 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.631979979 +0000 2024-04-17T06:11:00.200 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.630979999 +0000 2024-04-17T06:11:00.200 INFO:tasks.cephfs.mount:File test_dir/test_file_98 became visible from 1 after 0s 2024-04-17T06:11:00.200 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2024-04-17T06:11:00.200 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:11:00.243 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.244 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.244 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.249 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.256 DEBUG:teuthology.orchestra.run.smithi190:> stdin-killer -- python3 -c ' 2024-04-17T06:11:00.256 DEBUG:teuthology.orchestra.run.smithi190:> import fcntl 2024-04-17T06:11:00.256 DEBUG:teuthology.orchestra.run.smithi190:> import os 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> import sys 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> import time 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> with open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_99", '"'"'r'"'"') as f: 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> while True: 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> print("open_background: keeping file open", file=sys.stderr) 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> try: 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> if os.read(0, 4096) == b"": 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> break 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> except BlockingIOError: 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> pass 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> time.sleep(2) 2024-04-17T06:11:00.257 DEBUG:teuthology.orchestra.run.smithi190:> ' 2024-04-17T06:11:00.300 DEBUG:teuthology.orchestra.run.smithi190:> stat /home/ubuntu/cephtest/mnt.1/test_dir/test_file_99 2024-04-17T06:11:00.308 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.317 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout: File: /home/ubuntu/cephtest/mnt.1/test_dir/test_file_99 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 2 Blocks: 1 IO Block: 4194304 regular file 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 2ah/42d Inode: 1099511628376 Links: 1 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0644/-rw-r--r--) Uid: ( 1000/ ubuntu) Gid: ( 1301/ ubuntu) 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2024-04-17 06:10:45.634979916 +0000 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2024-04-17 06:10:45.635979895 +0000 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2024-04-17 06:10:45.635979895 +0000 2024-04-17T06:11:00.319 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2024-04-17 06:10:45.634979916 +0000 2024-04-17T06:11:00.319 INFO:tasks.cephfs.mount:File test_dir/test_file_99 became visible from 1 after 0s 2024-04-17T06:11:00.320 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 counter dump 2024-04-17T06:11:00.362 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.363 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.364 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.364 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.428 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.429 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.438+0000 7f59ce738640 1 Processor -- start 2024-04-17T06:11:00.436 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.438+0000 7f59ce738640 1 -- start start 2024-04-17T06:11:00.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.438+0000 7f59ce738640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f59c815ff30 0x7f59c815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.438+0000 7f59ce738640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f59c8071ec0 0x7f59c8072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.437 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.438+0000 7f59ce738640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cf740 0x7f59c80cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.438+0000 7f59ce738640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f59c806df20 con 0x7f59c8071ec0 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.438+0000 7f59ce738640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f59c806eaa0 con 0x7f59c815ff30 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.438+0000 7f59ce738640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f59c806e170 con 0x7f59c80cf740 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c77fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f59c8071ec0 0x7f59c8072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c77fe640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f59c8071ec0 0x7f59c8072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53130/0 (socket says 172.21.15.135:53130) 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c77fe640 1 -- 172.21.15.135:0/1086019480 learned_addr learned my addr 172.21.15.135:0/1086019480 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59cccd5640 1 --2- 172.21.15.135:0/1086019480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cf740 0x7f59c80cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c7fff640 1 --2- 172.21.15.135:0/1086019480 >> v2:172.21.15.190:3300/0 conn(0x7f59c815ff30 0x7f59c815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c77fe640 1 -- 172.21.15.135:0/1086019480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cf740 msgr2=0x7f59c80cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c77fe640 1 --2- 172.21.15.135:0/1086019480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cf740 0x7f59c80cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c77fe640 1 -- 172.21.15.135:0/1086019480 >> v2:172.21.15.190:3300/0 conn(0x7f59c815ff30 msgr2=0x7f59c815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c77fe640 1 --2- 172.21.15.135:0/1086019480 >> v2:172.21.15.190:3300/0 conn(0x7f59c815ff30 0x7f59c815cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.438 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c77fe640 1 -- 172.21.15.135:0/1086019480 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c806e450 con 0x7f59c8071ec0 2024-04-17T06:11:00.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59cccd5640 1 --2- 172.21.15.135:0/1086019480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cf740 0x7f59c80cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:00.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.439+0000 7f59c7fff640 1 --2- 172.21.15.135:0/1086019480 >> v2:172.21.15.190:3300/0 conn(0x7f59c815ff30 0x7f59c815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:00.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59c77fe640 1 --2- 172.21.15.135:0/1086019480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c8072340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59c80b3350 tx=0x7f59bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cb1a30491fca2e server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59c6ffd640 1 -- 172.21.15.135:0/1086019480 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc035020 con 0x7f59c8071ec0 2024-04-17T06:11:00.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59c6ffd640 1 -- 172.21.15.135:0/1086019480 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc031870 con 0x7f59c8071ec0 2024-04-17T06:11:00.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59c6ffd640 1 -- 172.21.15.135:0/1086019480 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc031c40 con 0x7f59c8071ec0 2024-04-17T06:11:00.442 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59ce738640 1 -- 172.21.15.135:0/1086019480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 msgr2=0x7f59c8072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59ce738640 1 --2- 172.21.15.135:0/1086019480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c8072340 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59c80b3350 tx=0x7f59bc02b910 comp rx=0 tx=0).stop 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59ce738640 1 -- 172.21.15.135:0/1086019480 shutdown_connections 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59ce738640 1 --2- 172.21.15.135:0/1086019480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80cf740 0x7f59c80cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59ce738640 1 --2- 172.21.15.135:0/1086019480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c8072340 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59ce738640 1 --2- 172.21.15.135:0/1086019480 >> v2:172.21.15.190:3300/0 conn(0x7f59c815ff30 0x7f59c815cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.440+0000 7f59ce738640 1 -- 172.21.15.135:0/1086019480 >> 172.21.15.135:0/1086019480 conn(0x7f59c80d0520 msgr2=0x7f59c80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.441+0000 7f59ce738640 1 -- 172.21.15.135:0/1086019480 shutdown_connections 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.441+0000 7f59ce738640 1 -- 172.21.15.135:0/1086019480 wait complete. 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.442+0000 7f59ce738640 1 Processor -- start 2024-04-17T06:11:00.443 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.442+0000 7f59ce738640 1 -- start start 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.442+0000 7f59ce738640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c80cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59ce738640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c80cf740 0x7f59c8139150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59ce738640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80a9dd0 0x7f59c81366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59ce738640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80d0290 con 0x7f59c8071ec0 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59ce738640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80d0110 con 0x7f59c80cf740 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59ce738640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f59c8160390 con 0x7f59c80a9dd0 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c80cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c80cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53134/0 (socket says 172.21.15.135:53134) 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 -- 172.21.15.135:0/3305899087 learned_addr learned my addr 172.21.15.135:0/3305899087 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c77fe640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c80cf740 0x7f59c8139150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59cccd5640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80a9dd0 0x7f59c81366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 -- 172.21.15.135:0/3305899087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80a9dd0 msgr2=0x7f59c81366c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80a9dd0 0x7f59c81366c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 -- 172.21.15.135:0/3305899087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c80cf740 msgr2=0x7f59c8139150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.444 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c80cf740 0x7f59c8139150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 -- 172.21.15.135:0/3305899087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c8139690 con 0x7f59c8071ec0 2024-04-17T06:11:00.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c7fff640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c80cb660 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c7f0 tx=0x7f59b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c4ff9640 1 -- 172.21.15.135:0/3305899087 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8015070 con 0x7f59c8071ec0 2024-04-17T06:11:00.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c4ff9640 1 -- 172.21.15.135:0/3305899087 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b80077c0 con 0x7f59c8071ec0 2024-04-17T06:11:00.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59ce738640 1 -- 172.21.15.135:0/3305899087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c80cbdd0 con 0x7f59c8071ec0 2024-04-17T06:11:00.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59c4ff9640 1 -- 172.21.15.135:0/3305899087 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8007b70 con 0x7f59c8071ec0 2024-04-17T06:11:00.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.443+0000 7f59ce738640 1 -- 172.21.15.135:0/3305899087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c80cc390 con 0x7f59c8071ec0 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.445+0000 7f59c4ff9640 1 -- 172.21.15.135:0/3305899087 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59b8018070 con 0x7f59c8071ec0 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.445+0000 7f59c4ff9640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59b406c920 0x7f59b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.446+0000 7f59c4ff9640 1 -- 172.21.15.135:0/3305899087 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6130+0+0 (secure 0 0 0) 0x7f59b808ba50 con 0x7f59c8071ec0 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.446+0000 7f59c77fe640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59b406c920 0x7f59b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.446+0000 7f59c77fe640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59b406c920 0x7f59b406ede0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59bc036000 tx=0x7f59bc03c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59d13f3740 1 Processor -- start 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59d13f3740 1 -- start start 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb560380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb590170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5608c0 0x55fceb5906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59d13f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fceb528030 con 0x55fceb58d3e0 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59d13f3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fceb4cb100 con 0x55fceb58d7c0 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59d13f3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55fceb468570 con 0x55fceb5608c0 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb560380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb560380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53148/0 (socket says 172.21.15.135:53148) 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.447+0000 7f59c7fff640 1 -- 172.21.15.135:0/1579168484 learned_addr learned my addr 172.21.15.135:0/1579168484 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59c77fe640 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb590170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59c7fff640 1 -- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5608c0 msgr2=0x55fceb5906b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59cccd5640 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5608c0 0x55fceb5906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59c7fff640 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5608c0 0x55fceb5906b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59c7fff640 1 -- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 msgr2=0x55fceb590170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59c7fff640 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb590170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59c7fff640 1 -- 172.21.15.135:0/1579168484 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fceb483ed0 con 0x55fceb58d3e0 2024-04-17T06:11:00.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59c77fe640 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb590170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59c7fff640 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb560380 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f59b8061c80 tx=0x7f59b805dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c093b2833fadfe server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59adffb640 1 -- 172.21.15.135:0/1579168484 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b801f040 con 0x55fceb58d3e0 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59adffb640 1 -- 172.21.15.135:0/1579168484 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8055cf0 con 0x55fceb58d3e0 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.448+0000 7f59adffb640 1 -- 172.21.15.135:0/1579168484 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8058cd0 con 0x55fceb58d3e0 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 -- 172.21.15.135:0/1579168484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 msgr2=0x55fceb560380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb560380 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f59b8061c80 tx=0x7f59b805dac0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 -- 172.21.15.135:0/1579168484 shutdown_connections 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5608c0 0x55fceb5906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb590170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1579168484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb560380 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 -- 172.21.15.135:0/1579168484 >> 172.21.15.135:0/1579168484 conn(0x55fceb5625f0 msgr2=0x55fceb5ab170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 -- 172.21.15.135:0/1579168484 shutdown_connections 2024-04-17T06:11:00.456 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 -- 172.21.15.135:0/1579168484 wait complete. 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.449+0000 7f59d13f3740 1 -- start start 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59d13f3740 1 Processor -- start 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d3e0 0x55fceb58b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb594b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb5bba20 0x55fceb5bbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59d13f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fceb484e80 con 0x55fceb5bba20 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59d13f3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fceb4c70c0 con 0x55fceb58d7c0 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59d13f3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55fceb564c90 con 0x55fceb58d3e0 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59d13f3740 10 client.0 _ll_register_callbacks cb 0x55fceb4c1560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59cccd5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb5bba20 0x55fceb5bbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59cccd5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb5bba20 0x55fceb5bbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53152/0 (socket says 172.21.15.135:53152) 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59cccd5640 1 -- 172.21.15.135:0/4002533539 learned_addr learned my addr 172.21.15.135:0/4002533539 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.450+0000 7f59c77fe640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb594b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.457 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59c7fff640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d3e0 0x55fceb58b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59cccd5640 1 -- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d3e0 msgr2=0x55fceb58b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59cccd5640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d3e0 0x55fceb58b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59cccd5640 1 -- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 msgr2=0x55fceb594b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59cccd5640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb594b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59cccd5640 1 -- 172.21.15.135:0/4002533539 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fceb1dd3f0 con 0x55fceb5bba20 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59c7fff640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d3e0 0x55fceb58b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59c77fe640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb594b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59cccd5640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb5bba20 0x55fceb5bbea0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f59c000ebb0 tx=0x7f59c000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfd239a0a681f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59a6ffd640 1 -- 172.21.15.135:0/4002533539 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c001c060 con 0x55fceb5bba20 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59a6ffd640 1 -- 172.21.15.135:0/4002533539 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6130+0+0 (secure 0 0 0) 0x7f59c000a620 con 0x55fceb5bba20 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59d13f3740 1 -- 172.21.15.135:0/4002533539 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b8055cf0 con 0x55fceb5bba20 2024-04-17T06:11:00.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59a6ffd640 1 -- 172.21.15.135:0/4002533539 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f59c000a820 con 0x55fceb5bba20 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59d13f3740 10 client.5625 fetch_fsmap learned FSMap version 16 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59d13f3740 1 -- 172.21.15.135:0/4002533539 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55fceb2a77d0 con 0x55fceb5bba20 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.451+0000 7f59a6ffd640 1 -- 172.21.15.135:0/4002533539 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8055cf0 con 0x55fceb5bba20 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59a6ffd640 1 -- 172.21.15.135:0/4002533539 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f59c0025080 con 0x55fceb5bba20 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59d13f3740 10 client.5625 fetch_fsmap finished waiting for FSMap version 16 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59d13f3740 10 client.5625 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59d13f3740 20 client.5625 populate_metadata read hostname 'smithi135' 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59d13f3740 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x55fceb5921c0 0x55fceb5bf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59d13f3740 4 client.5625 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59d13f3740 1 -- 172.21.15.135:0/4002533539 --> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fceb470f30 con 0x55fceb5921c0 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59c7fff640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x55fceb5921c0 0x55fceb5bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59a6ffd640 10 client.5625 ms_handle_connect on v2:172.21.15.135:6835/3309431937 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.452+0000 7f59c7fff640 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x55fceb5921c0 0x55fceb5bf040 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739b2089cc7fc4b8 server_cookie=eac734bdcd95fa3c in_seq=0 out_seq=0 2024-04-17T06:11:00.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59a6ffd640 1 -- 172.21.15.135:0/4002533539 <== mds.0 v2:172.21.15.135:6835/3309431937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fceb470f30 con 0x55fceb5921c0 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59a6ffd640 10 client.5625 handle_command_reply: tid=0 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 client.5625 shutdown 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 -- 172.21.15.135:0/4002533539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb5bba20 msgr2=0x55fceb5bbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb5bba20 0x55fceb5bbea0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f59c000ebb0 tx=0x7f59c000c7f0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 -- 172.21.15.135:0/4002533539 shutdown_connections 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb5bba20 0x55fceb5bbea0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x55fceb5921c0 0x55fceb5bf040 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb594b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 --2- 172.21.15.135:0/4002533539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d3e0 0x55fceb58b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59a6ffd640 0 client.5625 ms_handle_reset on v2:172.21.15.135:6835/3309431937 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 -- 172.21.15.135:0/4002533539 >> 172.21.15.135:0/4002533539 conn(0x55fceb5608c0 msgr2=0x55fceb5ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 -- 172.21.15.135:0/4002533539 shutdown_connections 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 1 -- 172.21.15.135:0/4002533539 wait complete. 2024-04-17T06:11:00.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.453+0000 7f59d13f3740 20 client.5625 trim_cache size 0 max 16384 2024-04-17T06:11:00.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.466+0000 7f59d13f3740 1 Processor -- start 2024-04-17T06:11:00.464 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.466+0000 7f59d13f3740 1 -- start start 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb5ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d7c0 0x55fceb5902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb5bba20 0x55fceb5ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59d13f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8201550 con 0x55fceb58d3e0 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59d13f3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8200590 con 0x55fceb5bba20 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59d13f3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f59c80b4c50 con 0x55fceb58d7c0 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb5ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb5ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53166/0 (socket says 172.21.15.135:53166) 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c7fff640 1 -- 172.21.15.135:0/3863086130 learned_addr learned my addr 172.21.15.135:0/3863086130 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:00.467 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c7fff640 1 -- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d7c0 msgr2=0x55fceb5902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c77fe640 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d7c0 0x55fceb5902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59cccd5640 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb5bba20 0x55fceb5ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c7fff640 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d7c0 0x55fceb5902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c7fff640 1 -- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb5bba20 msgr2=0x55fceb5ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c7fff640 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb5bba20 0x55fceb5ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c7fff640 1 -- 172.21.15.135:0/3863086130 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fceb5aa4f0 con 0x55fceb58d3e0 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59c77fe640 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d7c0 0x55fceb5902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.467+0000 7f59cccd5640 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb5bba20 0x55fceb5ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59c7fff640 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb5ad930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c7c0 tx=0x7f59b801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58432bbfe2f9550 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59ad7fa640 1 -- 172.21.15.135:0/3863086130 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b801f040 con 0x55fceb58d3e0 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59ad7fa640 1 -- 172.21.15.135:0/3863086130 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8076070 con 0x55fceb58d3e0 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59ad7fa640 1 -- 172.21.15.135:0/3863086130 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8055cf0 con 0x55fceb58d3e0 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 -- 172.21.15.135:0/3863086130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 msgr2=0x55fceb5ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.468 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb5ad930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c7c0 tx=0x7f59b801c000 comp rx=0 tx=0).stop 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 -- 172.21.15.135:0/3863086130 shutdown_connections 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb5bba20 0x55fceb5ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb58d7c0 0x55fceb5902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 --2- 172.21.15.135:0/3863086130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb5ad930 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 -- 172.21.15.135:0/3863086130 >> 172.21.15.135:0/3863086130 conn(0x55fceb5608c0 msgr2=0x55fceb58b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 -- 172.21.15.135:0/3863086130 shutdown_connections 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.468+0000 7f59d13f3740 1 -- 172.21.15.135:0/3863086130 wait complete. 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.469+0000 7f59d13f3740 1 -- start start 2024-04-17T06:11:00.469 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.469+0000 7f59d13f3740 1 Processor -- start 2024-04-17T06:11:00.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb625060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb5ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.473 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59d13f3740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5bba20 0x55fceb5aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59d13f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81fff70 con 0x55fceb58d3e0 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59d13f3740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x55fceb5ab7f0 con 0x55fceb58d7c0 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59d13f3740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x55fceb58c630 con 0x55fceb5bba20 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59d13f3740 10 client.0 _ll_register_callbacks cb 0x7f59c0007a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb625060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb625060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53178/0 (socket says 172.21.15.135:53178) 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 -- 172.21.15.135:0/1367038195 learned_addr learned my addr 172.21.15.135:0/1367038195 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c77fe640 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb5ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59cccd5640 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5bba20 0x55fceb5aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 -- 172.21.15.135:0/1367038195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5bba20 msgr2=0x55fceb5aa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5bba20 0x55fceb5aa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 -- 172.21.15.135:0/1367038195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 msgr2=0x55fceb5ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb5ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 -- 172.21.15.135:0/1367038195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fceb563af0 con 0x55fceb58d3e0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59c7fff640 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb625060 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c7c0 tx=0x7f59b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73cee2ea581afbc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.470+0000 7f59a6ffd640 1 -- 172.21.15.135:0/1367038195 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b807e060 con 0x55fceb58d3e0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59a6ffd640 1 -- 172.21.15.135:0/1367038195 <== mon.0 v2:172.21.15.135:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6130+0+0 (secure 0 0 0) 0x7f59b805eb10 con 0x55fceb58d3e0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 1 -- 172.21.15.135:0/1367038195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b8076070 con 0x55fceb58d3e0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59a6ffd640 1 -- 172.21.15.135:0/1367038195 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f59b801f040 con 0x55fceb58d3e0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 10 client.5631 fetch_fsmap learned FSMap version 16 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 1 -- 172.21.15.135:0/1367038195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55fceb5919c0 con 0x55fceb58d3e0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59a6ffd640 1 -- 172.21.15.135:0/1367038195 <== mon.0 v2:172.21.15.135:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b8076070 con 0x55fceb58d3e0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59a6ffd640 1 -- 172.21.15.135:0/1367038195 <== mon.0 v2:172.21.15.135:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7f59b80553d0 con 0x55fceb58d3e0 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 10 client.5631 fetch_fsmap finished waiting for FSMap version 16 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 10 client.5631 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 20 client.5631 populate_metadata read hostname 'smithi135' 2024-04-17T06:11:00.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x55fceb60d860 0x55fceb5aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 4 client.5631 mds_command: new command op to 5379 tid=0 multi_id=0 [{"prefix": "counter dump"}] 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59d13f3740 1 -- 172.21.15.135:0/1367038195 --> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] -- command(tid 0: {"prefix": "counter dump"}) -- 0x55fceb563cc0 con 0x55fceb60d860 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59c77fe640 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x55fceb60d860 0x55fceb5aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59a6ffd640 10 client.5631 ms_handle_connect on v2:172.21.15.135:6835/3309431937 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.471+0000 7f59c77fe640 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x55fceb60d860 0x55fceb5aaf30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f19e6e45ecbdd1 server_cookie=631a0ab279021cff in_seq=0 out_seq=0 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59a6ffd640 1 -- 172.21.15.135:0/1367038195 <== mds.0 v2:172.21.15.135:6835/3309431937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+27031 (crc 0 0 0) 0x55fceb563cc0 con 0x55fceb60d860 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59a6ffd640 10 client.5631 handle_command_reply: tid=0 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 client.5631 shutdown 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 -- 172.21.15.135:0/1367038195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 msgr2=0x55fceb625060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb625060 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c7c0 tx=0x7f59b80046c0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 -- 172.21.15.135:0/1367038195 shutdown_connections 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55fceb5bba20 0x55fceb5aa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.135:6835/3309431937,v1:172.21.15.135:6837/3309431937] conn(0x55fceb60d860 0x55fceb5aaf30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.476 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55fceb58d7c0 0x55fceb5ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 --2- 172.21.15.135:0/1367038195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fceb58d3e0 0x55fceb625060 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 -- 172.21.15.135:0/1367038195 >> 172.21.15.135:0/1367038195 conn(0x55fceb5608c0 msgr2=0x55fceb566ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:00.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59a6ffd640 0 client.5631 ms_handle_reset on v2:172.21.15.135:6835/3309431937 2024-04-17T06:11:00.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 -- 172.21.15.135:0/1367038195 shutdown_connections 2024-04-17T06:11:00.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 1 -- 172.21.15.135:0/1367038195 wait complete. 2024-04-17T06:11:00.477 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.472+0000 7f59d13f3740 20 client.5631 trim_cache size 0 max 16384 2024-04-17T06:11:00.477 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker": [ 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "0" 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "1" 2024-04-17T06:11:00.478 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "2" 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_ready_timeouts": 0, 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_connection_idle_timeouts": 0 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.479 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": [ 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1320, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1303, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 331825, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 4774718, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 8, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.087564154, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.037792702, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.027303709, 2024-04-17T06:11:00.480 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.020167575, 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1302, 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.204594737, 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000157138 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 254, 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000098014, 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000385 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.481 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": [ 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 117, 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 124, 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 28932, 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 455928, 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 12, 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 9, 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.019497801, 2024-04-17T06:11:00.482 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.003693696, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.012171382, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.002714629, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 123, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.133342573, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001084085 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 6, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000396, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000066 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 99232 2024-04-17T06:11:00.483 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": [ 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 3271, 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 4274, 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 911217, 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 1836167, 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 7, 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.195658117, 2024-04-17T06:11:00.484 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.129002160, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.117249454, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.001593729, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4274, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.081812985, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000019142 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4207, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000649013, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000154 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 114960, 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 15104 2024-04-17T06:11:00.485 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": [ 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": [ 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.486 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2263, 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.612270382, 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000270556 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": [ 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.487 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000015348, 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000007674 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": [ 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 1247, 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 1247, 2024-04-17T06:11:00.488 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1247, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1.026639986, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000823287 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 12, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 1012, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 0, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 1012, 2024-04-17T06:11:00.489 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 1011, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 1002, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 1104, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 2, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 1491, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 1488, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-04-17T06:11:00.490 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 1000, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 2890, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-04-17T06:11:00.491 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 167, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 1044, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 1173, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 1000, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-04-17T06:11:00.492 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 3 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": [ 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-04-17T06:11:00.493 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-04-17T06:11:00.494 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_metrics": [ 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: "fs_name": "cephfs", 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: "id": "5379" 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.495 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "num_clients": 2 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_client_metrics-cephfs": [ 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.4943", 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "rank": "0" 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_hits": 689, 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_miss": 94, 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_read_latency": 0.000000000, 2024-04-17T06:11:00.496 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_write_latency": 0.000000000, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_metadata_latency": 0.000000000, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_hits": 411, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_miss": 0, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_files": 82, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_inodes": 84, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pinned_icaps": 84, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "total_inodes": 84, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_ops": 0, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_size": 0, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_ops": 0, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_size": 0 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.497 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": { 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "client": "client.5574", 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "rank": "0" 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_hits": 9034, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_miss": 20, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_read_latency": 0.000000000, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_write_latency": 0.000000000, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_metadata_latency": 0.000000000, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_hits": 1002, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "dentry_lease_miss": 0, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_files": 0, 2024-04-17T06:11:00.498 INFO:teuthology.orchestra.run.smithi135.stdout: "opened_inodes": 1002, 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "pinned_icaps": 1002, 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "total_inodes": 1002, 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_ops": 0, 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "total_read_size": 0, 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_ops": 1000, 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "total_write_size": 2890 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": [ 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.499 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 2104, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 2104, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 4, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 4, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 2, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-04-17T06:11:00.500 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 8586060, 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1100, 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1.515783124, 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001377984 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-04-17T06:11:00.501 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": [ 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 1014, 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 1014, 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 0, 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 13, 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 13, 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 0, 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 1012, 2024-04-17T06:11:00.502 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 1012, 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 0, 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 1104, 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 1104, 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 56916, 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 231784 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": [ 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.503 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 1484, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 1247, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 1004, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1000, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.686096504, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000686096 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 142, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.229404901, 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001615527 2024-04-17T06:11:00.504 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 101, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.107630518, 2024-04-17T06:11:00.505 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001065648 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.506 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-04-17T06:11:00.507 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.003037913, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003037913 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.508 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000203295, 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000101647 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.510 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.511 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000266855, 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000266855 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-04-17T06:11:00.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.514 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": [ 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 2, 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 2, 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 2, 2024-04-17T06:11:00.515 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 1085, 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 542, 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 21, 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": [ 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-04-17T06:11:00.516 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-04-17T06:11:00.517 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-04-17T06:11:00.518 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 296230, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 1122, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 1672, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 19, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-04-17T06:11:00.519 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 5424, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 76, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 3578593, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 40340, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.520 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": [ 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 1156, 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 4472286, 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 37, 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 1156, 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1156, 2024-04-17T06:11:00.521 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 2.329951634, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002015529 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1156, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1195 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 1156, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 0, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 1156, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-04-17T06:11:00.522 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 11, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 1100, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 9, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 7, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 0, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 22, 2024-04-17T06:11:00.523 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-04-17T06:11:00.524 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 39, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-04-17T06:11:00.525 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 2, 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 8, 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 8, 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 35, 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 11 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": [ 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.526 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": [ 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.527 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": [ 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.528 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": [ 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 4702, 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 899676, 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 4702, 2024-04-17T06:11:00.529 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4702, 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 899676, 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": [ 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.530 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 1156, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 4472286, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1156, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 4472286, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.531 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": [ 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:11:00.532 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 1156, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 1156, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1156, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1156, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.533 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": [ 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:11:00.534 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: ], 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x56392f628500": [ 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: { 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "labels": {}, 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "counters": { 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-04-17T06:11:00.535 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 2104, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 4391756, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 2104, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 1100, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 4391756, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.536 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.537 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-04-17T06:11:00.537 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-04-17T06:11:00.537 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-04-17T06:11:00.538 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.539 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 -- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59b406c920 msgr2=0x7f59b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59b406c920 0x7f59b406ede0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59bc036000 tx=0x7f59bc03c040 comp rx=0 tx=0).stop 2024-04-17T06:11:00.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 -- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 msgr2=0x7f59c80cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c80cb660 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f59b800c7f0 tx=0x7f59b800ccb0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 -- 172.21.15.135:0/3305899087 shutdown_connections 2024-04-17T06:11:00.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59c80a9dd0 0x7f59c81366c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.539 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59c80cf740 0x7f59c8139150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f59b406c920 0x7f59b406ede0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 --2- 172.21.15.135:0/3305899087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59c8071ec0 0x7f59c80cb660 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 -- 172.21.15.135:0/3305899087 >> 172.21.15.135:0/3305899087 conn(0x7f59c80d0520 msgr2=0x7f59c81449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:00.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 -- 172.21.15.135:0/3305899087 shutdown_connections 2024-04-17T06:11:00.540 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.473+0000 7f59ce738640 1 -- 172.21.15.135:0/3305899087 wait complete. 2024-04-17T06:11:00.540 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.540 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T06:11:00.541 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:11:00.541 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:11:00.546 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.547 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.557 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-04-17T06:11:00.557 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-17T06:11:00.598 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.600 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.605 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.613 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-17T06:11:00.665 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.665 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.717 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.718 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.722 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.755 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.757+0000 7f7aafbd4640 1 Processor -- start 2024-04-17T06:11:00.756 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.758+0000 7f7aafbd4640 1 -- start start 2024-04-17T06:11:00.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.758+0000 7f7aafbd4640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7aa80d19e0 0x7f7aa81696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.758+0000 7f7aafbd4640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa80d30a0 con 0x7f7aa80d43d0 2024-04-17T06:11:00.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.758+0000 7f7aafbd4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa80d3330 con 0x7f7aa815ec00 2024-04-17T06:11:00.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.758+0000 7f7aafbd4640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa815efe0 con 0x7f7aa80d19e0 2024-04-17T06:11:00.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.759+0000 7f7aae171640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7aa80d19e0 0x7f7aa81696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.759+0000 7f7aad16f640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f7aa80d43d0 0x7f7aa80d12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39104/0 (socket says 172.21.15.135:39104) 2024-04-17T06:11:00.757 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.759+0000 7f7aad16f640 1 -- 172.21.15.135:0/1159174933 learned_addr learned my addr 172.21.15.135:0/1159174933 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:11:00.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.759+0000 7f7aae171640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f7aa80d19e0 0x7f7aa81696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53190/0 (socket says 172.21.15.135:53190) 2024-04-17T06:11:00.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.759+0000 7f7aae171640 1 -- 172.21.15.135:0/1159174933 >> v1:172.21.15.190:6789/0 conn(0x7f7aa815ec00 legacy=0x7f7aa80d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:00.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.759+0000 7f7aae171640 1 -- 172.21.15.135:0/1159174933 >> v1:172.21.15.135:6789/0 conn(0x7f7aa80d43d0 legacy=0x7f7aa80d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.759+0000 7f7aae171640 1 -- 172.21.15.135:0/1159174933 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa80d3610 con 0x7f7aa80d19e0 2024-04-17T06:11:00.758 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.759+0000 7f7aac96e640 1 -- 172.21.15.135:0/1159174933 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2199720855 0 0) 0x7f7aa80d30a0 con 0x7f7aa80d43d0 2024-04-17T06:11:00.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aae171640 1 --2- 172.21.15.135:0/1159174933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa81696c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009d90 tx=0x7f7aa002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad705aaea10e0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aac96e640 1 -- 172.21.15.135:0/1159174933 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0039070 con 0x7f7aa80d19e0 2024-04-17T06:11:00.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aac96e640 1 -- 172.21.15.135:0/1159174933 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa002cd00 con 0x7f7aa80d19e0 2024-04-17T06:11:00.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aac96e640 1 -- 172.21.15.135:0/1159174933 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0034660 con 0x7f7aa80d19e0 2024-04-17T06:11:00.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aafbd4640 1 -- 172.21.15.135:0/1159174933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 msgr2=0x7f7aa81696c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aafbd4640 1 --2- 172.21.15.135:0/1159174933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa81696c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009d90 tx=0x7f7aa002bf50 comp rx=0 tx=0).stop 2024-04-17T06:11:00.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aafbd4640 1 -- 172.21.15.135:0/1159174933 shutdown_connections 2024-04-17T06:11:00.759 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aafbd4640 1 --2- 172.21.15.135:0/1159174933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa81696c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aafbd4640 1 -- 172.21.15.135:0/1159174933 >> 172.21.15.135:0/1159174933 conn(0x7f7aa806cc90 msgr2=0x7f7aa806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:00.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aafbd4640 1 -- 172.21.15.135:0/1159174933 shutdown_connections 2024-04-17T06:11:00.760 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.760+0000 7f7aafbd4640 1 -- 172.21.15.135:0/1159174933 wait complete. 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aafbd4640 1 Processor -- start 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aafbd4640 1 -- start start 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aafbd4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aafbd4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa80d43d0 0x7f7aa81f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aafbd4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815ec00 0x7f7aa81f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aafbd4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa80d3bd0 con 0x7f7aa80d19e0 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aafbd4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa80d3f40 con 0x7f7aa80d43d0 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aafbd4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa80ba6e0 con 0x7f7aa815ec00 2024-04-17T06:11:00.763 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aad970640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aad970640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa815fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53198/0 (socket says 172.21.15.135:53198) 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.762+0000 7f7aad970640 1 -- 172.21.15.135:0/946253887 learned_addr learned my addr 172.21.15.135:0/946253887 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aad16f640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa80d43d0 0x7f7aa81f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aae171640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815ec00 0x7f7aa81f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aad970640 1 -- 172.21.15.135:0/946253887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815ec00 msgr2=0x7f7aa81f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aad970640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815ec00 0x7f7aa81f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aad970640 1 -- 172.21.15.135:0/946253887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa80d43d0 msgr2=0x7f7aa81f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aad970640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa80d43d0 0x7f7aa81f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aad970640 1 -- 172.21.15.135:0/946253887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa80ca8e0 con 0x7f7aa80d19e0 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aad970640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa815fb70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400b3f0 tx=0x7f7aa400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7a9e7fc640 1 -- 172.21.15.135:0/946253887 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4015020 con 0x7f7aa80d19e0 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7a9e7fc640 1 -- 172.21.15.135:0/946253887 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa4011830 con 0x7f7aa80d19e0 2024-04-17T06:11:00.764 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa80cabc0 con 0x7f7aa80d19e0 2024-04-17T06:11:00.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7a9e7fc640 1 -- 172.21.15.135:0/946253887 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4011c20 con 0x7f7aa80d19e0 2024-04-17T06:11:00.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.763+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa81f48f0 con 0x7f7aa80d19e0 2024-04-17T06:11:00.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.764+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa808c540 con 0x7f7aa80d19e0 2024-04-17T06:11:00.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.765+0000 7f7a9e7fc640 1 -- 172.21.15.135:0/946253887 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7aa4018020 con 0x7f7aa80d19e0 2024-04-17T06:11:00.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.765+0000 7f7a9e7fc640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7a8c06c8f0 0x7f7a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:00.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.765+0000 7f7a9e7fc640 1 -- 172.21.15.135:0/946253887 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6130+0+0 (secure 0 0 0) 0x7f7aa408b760 con 0x7f7aa80d19e0 2024-04-17T06:11:00.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.765+0000 7f7aad16f640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7a8c06c8f0 0x7f7a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:00.765 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.765+0000 7f7aad16f640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7a8c06c8f0 0x7f7a8c06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7aa815da80 tx=0x7f7a98007d50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:00.767 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.769+0000 7f7a9e7fc640 1 -- 172.21.15.135:0/946253887 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7aa808c540 con 0x7f7aa80d19e0 2024-04-17T06:11:00.784 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.784 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.836 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.839 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.840 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.902 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.902 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.946 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.948+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7aa8074940 con 0x7f7aa80d19e0 2024-04-17T06:11:00.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.949+0000 7f7a9e7fc640 1 -- 172.21.15.135:0/946253887 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17039 (secure 0 0 0) 0x7f7aa401f030 con 0x7f7aa80d19e0 2024-04-17T06:11:00.948 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-04-17T06:11:00.948 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":39,"fsid":"96728bf8-42ba-4aa8-88ba-489677bf3f41","created":"2024-04-17T06:08:55.708279+0000","modified":"2024-04-17T06:11:00.033946+0000","last_up_change":"2024-04-17T06:09:06.957582+0000","last_in_change":"2024-04-17T06:08:58.042387+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-17T06:09:07.953511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-17T06:10:25.616330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-17T06:10:26.077318+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4df49209-004c-4834-b05c-d4681c7e0f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6809","nonce":2300683224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6811","nonce":2300683224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6815","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6816","nonce":2300683224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":2300683224},{"type":"v1","addr":"172.21.15.135:6813","nonce":2300683224}]},"public_addr":"172.21.15.135:6809/2300683224","cluster_addr":"172.21.15.135:6811/2300683224","heartbeat_back_addr":"172.21.15.135:6816/2300683224","heartbeat_front_addr":"172.21.15.135:6813/2300683224","state":["exists","up"]},{"osd":1,"uuid":"a4181f82-fde6-4d13-96fb-3ddbf31a53ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6801","nonce":1452262953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6803","nonce":1452262953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6807","nonce":1452262953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":1452262953},{"type":"v1","addr":"172.21.15.135:6805","nonce":1452262953}]},"public_addr":"172.21.15.135:6801/1452262953","cluster_addr":"172.21.15.135:6803/1452262953","heartbeat_back_addr":"172.21.15.135:6807/1452262953","heartbeat_front_addr":"172.21.15.135:6805/1452262953","state":["exists","up"]},{"osd":2,"uuid":"250dc157-79ef-4459-906e-5ccc3b85ce94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6817","nonce":4143480345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6819","nonce":4143480345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6825","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6828","nonce":4143480345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":4143480345},{"type":"v1","addr":"172.21.15.135:6823","nonce":4143480345}]},"public_addr":"172.21.15.135:6817/4143480345","cluster_addr":"172.21.15.135:6819/4143480345","heartbeat_back_addr":"172.21.15.135:6828/4143480345","heartbeat_front_addr":"172.21.15.135:6823/4143480345","state":["exists","up"]},{"osd":3,"uuid":"5719faec-5aaf-4944-87d0-aafafb5107bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6821","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6822","nonce":4267051868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6826","nonce":4267051868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6831","nonce":4267051868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6827","nonce":4267051868},{"type":"v1","addr":"172.21.15.135:6829","nonce":4267051868}]},"public_addr":"172.21.15.135:6822/4267051868","cluster_addr":"172.21.15.135:6826/4267051868","heartbeat_back_addr":"172.21.15.135:6831/4267051868","heartbeat_front_addr":"172.21.15.135:6829/4267051868","state":["exists","up"]},{"osd":4,"uuid":"970a8be4-0bb2-46ad-8e4e-127c3d58b2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6801","nonce":4084657614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6803","nonce":4084657614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6807","nonce":4084657614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":4084657614},{"type":"v1","addr":"172.21.15.190:6805","nonce":4084657614}]},"public_addr":"172.21.15.190:6801/4084657614","cluster_addr":"172.21.15.190:6803/4084657614","heartbeat_back_addr":"172.21.15.190:6807/4084657614","heartbeat_front_addr":"172.21.15.190:6805/4084657614","state":["exists","up"]},{"osd":5,"uuid":"79cd6222-7c39-4b17-bf01-253339bd1678","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6817","nonce":1825459581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6819","nonce":1825459581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6825","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6827","nonce":1825459581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6821","nonce":1825459581},{"type":"v1","addr":"172.21.15.190:6823","nonce":1825459581}]},"public_addr":"172.21.15.190:6817/1825459581","cluster_addr":"172.21.15.190:6819/1825459581","heartbeat_back_addr":"172.21.15.190:6827/1825459581","heartbeat_front_addr":"172.21.15.190:6823/1825459581","state":["exists","up"]},{"osd":6,"uuid":"92a99e45-dcb9-43bc-98cc-650deff014ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6822","nonce":2045080651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6826","nonce":2045080651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6831","nonce":2045080651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":2045080651},{"type":"v1","addr":"172.21.15.190:6829","nonce":2045080651}]},"public_addr":"172.21.15.190:6822/2045080651","cluster_addr":"172.21.15.190:6826/2045080651","heartbeat_back_addr":"172.21.15.190:6831/2045080651","heartbeat_front_addr":"172.21.15.190:6829/2045080651","state":["exists","up"]},{"osd":7,"uuid":"983a1cf1-3a3d-4ee3-9d88-eaa3ec57ade3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6809","nonce":163146865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6811","nonce":163146865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6815","nonce":163146865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":163146865},{"type":"v1","addr":"172.21.15.190:6813","nonce":163146865}]},"public_addr":"172.21.15.190:6809/163146865","cluster_addr":"172.21.15.190:6811/163146865","heartbeat_back_addr":"172.21.15.190:6815/163146865","heartbeat_front_addr":"172.21.15.190:6813/163146865","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.213340+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.133985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.286951+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.262391+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.100934+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.251549+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.267767+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-17T06:09:03.215146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.26","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-17T06:11:00.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7a8c06c8f0 msgr2=0x7f7a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7a8c06c8f0 0x7f7a8c06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7aa815da80 tx=0x7f7a98007d50 comp rx=0 tx=0).stop 2024-04-17T06:11:00.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 msgr2=0x7f7aa815fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:00.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa815fb70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400b3f0 tx=0x7f7aa400b8b0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 shutdown_connections 2024-04-17T06:11:00.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815ec00 0x7f7aa81f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa80d43d0 0x7f7aa81f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f7a8c06c8f0 0x7f7a8c06edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 --2- 172.21.15.135:0/946253887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7aa80d19e0 0x7f7aa815fb70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:00.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 >> 172.21.15.135:0/946253887 conn(0x7f7aa806cc90 msgr2=0x7f7aa80d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:00.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 shutdown_connections 2024-04-17T06:11:00.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:00.952+0000 7f7aafbd4640 1 -- 172.21.15.135:0/946253887 wait complete. 2024-04-17T06:11:00.956 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.957 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:00.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-17T06:09:07.953511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-17T06:10:25.616330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-17T06:10:26.077318+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-17T06:11:00.961 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-17T06:11:00.966 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.021 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.022 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.072+0000 7fb94c53f640 1 Processor -- start 2024-04-17T06:11:01.070 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.072+0000 7fb94c53f640 1 -- start start 2024-04-17T06:11:01.071 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94c53f640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb94406caa0 0x7fb94406ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94c53f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9440b3e30 0x7fb9440b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94c53f640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb94406d3c0 con 0x7fb94415ba20 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94c53f640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7fb9440b5730 con 0x7fb94406caa0 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94c53f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb9440b4b20 con 0x7fb9440b3e30 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94a2db640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7fb94406caa0 0x7fb94406ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94aadc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fb94415ba20 0x7fb94415beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39112/0 (socket says 172.21.15.135:39112) 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94aadc640 1 -- 172.21.15.135:0/3153100775 learned_addr learned my addr 172.21.15.135:0/3153100775 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94a2db640 1 -- 172.21.15.135:0/3153100775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9440b3e30 msgr2=0x7fb9440b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94a2db640 1 --2- 172.21.15.135:0/3153100775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9440b3e30 0x7fb9440b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94a2db640 1 -- 172.21.15.135:0/3153100775 >> v1:172.21.15.135:6789/0 conn(0x7fb94415ba20 legacy=0x7fb94415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb94a2db640 1 -- 172.21.15.135:0/3153100775 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9440b4e00 con 0x7fb94406caa0 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.073+0000 7fb9492d9640 1 -- 172.21.15.135:0/3153100775 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 841404352 0 0) 0x7fb94406d3c0 con 0x7fb94415ba20 2024-04-17T06:11:01.072 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.074+0000 7fb94a2db640 1 --2- 172.21.15.135:0/3153100775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94406caa0 0x7fb94406ce80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb9400097a0 tx=0x7fb94002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913c58d25ce7c4de server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.074+0000 7fb9492d9640 1 -- 172.21.15.135:0/3153100775 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9400315a0 con 0x7fb94406caa0 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.074+0000 7fb9492d9640 1 -- 172.21.15.135:0/3153100775 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940037070 con 0x7fb94406caa0 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.074+0000 7fb9492d9640 1 -- 172.21.15.135:0/3153100775 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940031960 con 0x7fb94406caa0 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.074+0000 7fb94c53f640 1 -- 172.21.15.135:0/3153100775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94406caa0 msgr2=0x7fb94406ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.074+0000 7fb94c53f640 1 --2- 172.21.15.135:0/3153100775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94406caa0 0x7fb94406ce80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb9400097a0 tx=0x7fb94002b840 comp rx=0 tx=0).stop 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.075+0000 7fb94c53f640 1 -- 172.21.15.135:0/3153100775 shutdown_connections 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.075+0000 7fb94c53f640 1 --2- 172.21.15.135:0/3153100775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9440b3e30 0x7fb9440b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.075+0000 7fb94c53f640 1 --2- 172.21.15.135:0/3153100775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb94406caa0 0x7fb94406ce80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.074 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.075+0000 7fb94c53f640 1 -- 172.21.15.135:0/3153100775 >> 172.21.15.135:0/3153100775 conn(0x7fb9440cd7e0 msgr2=0x7fb9440cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:01.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.075+0000 7fb94c53f640 1 -- 172.21.15.135:0/3153100775 shutdown_connections 2024-04-17T06:11:01.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.075+0000 7fb94c53f640 1 -- 172.21.15.135:0/3153100775 wait complete. 2024-04-17T06:11:01.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.076+0000 7fb94c53f640 1 Processor -- start 2024-04-17T06:11:01.075 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.076+0000 7fb94c53f640 1 -- start start 2024-04-17T06:11:01.076 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.076+0000 7fb94c53f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb94406caa0 0x7fb94414a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.076+0000 7fb94c53f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9440b3e30 0x7fb94415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.076+0000 7fb94c53f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94415ba20 0x7fb94415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.076+0000 7fb94c53f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9440b47f0 con 0x7fb9440b3e30 2024-04-17T06:11:01.078 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.076+0000 7fb94c53f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fb9440b53c0 con 0x7fb94406caa0 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.076+0000 7fb94c53f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fb9440705b0 con 0x7fb94415ba20 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9440b3e30 0x7fb94415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9440b3e30 0x7fb94415d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53216/0 (socket says 172.21.15.135:53216) 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 -- 172.21.15.135:0/1017160009 learned_addr learned my addr 172.21.15.135:0/1017160009 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb94aadc640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94415ba20 0x7fb94415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb94a2db640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb94406caa0 0x7fb94414a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 -- 172.21.15.135:0/1017160009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94415ba20 msgr2=0x7fb94415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94415ba20 0x7fb94415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 -- 172.21.15.135:0/1017160009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb94406caa0 msgr2=0x7fb94414a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb94406caa0 0x7fb94414a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 -- 172.21.15.135:0/1017160009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb944150450 con 0x7fb9440b3e30 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb949ada640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9440b3e30 0x7fb94415d780 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b4a0 tx=0x7fb93400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb93affd640 1 -- 172.21.15.135:0/1017160009 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934014070 con 0x7fb9440b3e30 2024-04-17T06:11:01.079 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb93affd640 1 -- 172.21.15.135:0/1017160009 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb934010610 con 0x7fb9440b3e30 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9440b2450 con 0x7fb9440b3e30 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb93affd640 1 -- 172.21.15.135:0/1017160009 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934010a00 con 0x7fb9440b3e30 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.077+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9440b2730 con 0x7fb9440b3e30 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.078+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb944074940 con 0x7fb9440b3e30 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.079+0000 7fb93affd640 1 -- 172.21.15.135:0/1017160009 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb934017070 con 0x7fb9440b3e30 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.079+0000 7fb93affd640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb92806c8f0 0x7fb92806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.079+0000 7fb93affd640 1 -- 172.21.15.135:0/1017160009 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fb93408b610 con 0x7fb9440b3e30 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.079+0000 7fb94a2db640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb92806c8f0 0x7fb92806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.080+0000 7fb94a2db640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb92806c8f0 0x7fb92806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb940036000 tx=0x7fb94003d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.080 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.081 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.084+0000 7fb93affd640 1 -- 172.21.15.135:0/1017160009 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb9340592d0 con 0x7fb9440b3e30 2024-04-17T06:11:01.083 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.142 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.142 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.194 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.194 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.203 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.259 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.261 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.263+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb944153430 con 0x7fb9440b3e30 2024-04-17T06:11:01.261 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.263+0000 7fb93affd640 1 -- 172.21.15.135:0/1017160009 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7fb93405d5d0 con 0x7fb9440b3e30 2024-04-17T06:11:01.262 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 1 2024-04-17T06:11:01.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.265+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb92806c8f0 msgr2=0x7fb92806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.265+0000 7fb94c53f640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb92806c8f0 0x7fb92806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb940036000 tx=0x7fb94003d040 comp rx=0 tx=0).stop 2024-04-17T06:11:01.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.265+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9440b3e30 msgr2=0x7fb94415d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.264 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.265+0000 7fb94c53f640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9440b3e30 0x7fb94415d780 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b4a0 tx=0x7fb93400b960 comp rx=0 tx=0).stop 2024-04-17T06:11:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.266+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 shutdown_connections 2024-04-17T06:11:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.266+0000 7fb94c53f640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94415ba20 0x7fb94415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.266+0000 7fb94c53f640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fb92806c8f0 0x7fb92806edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.266+0000 7fb94c53f640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9440b3e30 0x7fb94415d780 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.266+0000 7fb94c53f640 1 --2- 172.21.15.135:0/1017160009 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb94406caa0 0x7fb94414a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.266+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 >> 172.21.15.135:0/1017160009 conn(0x7fb9440cd7e0 msgr2=0x7fb944152d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.266+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 shutdown_connections 2024-04-17T06:11:01.265 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.266+0000 7fb94c53f640 1 -- 172.21.15.135:0/1017160009 wait complete. 2024-04-17T06:11:01.275 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-17T06:11:01.311 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.314 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.321 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.377 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.378 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.383+0000 7fe956124640 1 Processor -- start 2024-04-17T06:11:01.381 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.383+0000 7fe956124640 1 -- start start 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe956124640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158a50 0x7fe95015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe956124640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe950158e30 0x7fe95006f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe956124640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe95015fb80 con 0x7fe9500b3590 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe956124640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7fe95006fb50 con 0x7fe950158e30 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe956124640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe95015b840 con 0x7fe950158a50 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94ffff640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7fe9500b3590 0x7fe95015bcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39118/0 (socket says 172.21.15.135:39118) 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94ffff640 1 -- 172.21.15.135:0/1389301955 learned_addr learned my addr 172.21.15.135:0/1389301955 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94effd640 1 --2- 172.21.15.135:0/1389301955 >> v2:172.21.15.190:3300/0 conn(0x7fe950158e30 0x7fe95006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94f7fe640 1 --2- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158a50 0x7fe95015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.383 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94e7fc640 1 -- 172.21.15.135:0/1389301955 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3315743460 0 0) 0x7fe95015fb80 con 0x7fe9500b3590 2024-04-17T06:11:01.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94e7fc640 1 -- 172.21.15.135:0/1389301955 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe938003520 con 0x7fe9500b3590 2024-04-17T06:11:01.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94effd640 1 -- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158a50 msgr2=0x7fe95015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:01.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94effd640 1 --2- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158a50 0x7fe95015ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94e7fc640 1 -- 172.21.15.135:0/1389301955 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2304348718 0 0) 0x7fe938003520 con 0x7fe9500b3590 2024-04-17T06:11:01.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94effd640 1 -- 172.21.15.135:0/1389301955 >> v1:172.21.15.135:6789/0 conn(0x7fe9500b3590 legacy=0x7fe95015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94effd640 1 -- 172.21.15.135:0/1389301955 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95015f2e0 con 0x7fe950158e30 2024-04-17T06:11:01.384 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.384+0000 7fe94f7fe640 1 --2- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158a50 0x7fe95015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-17T06:11:01.385 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.385+0000 7fe94effd640 1 --2- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158e30 0x7fe95006f610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe94400a940 tx=0x7fe94402cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7dc788477696d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.385+0000 7fe94e7fc640 1 -- 172.21.15.135:0/1389301955 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94403a070 con 0x7fe950158e30 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.385+0000 7fe94e7fc640 1 -- 172.21.15.135:0/1389301955 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94402d790 con 0x7fe950158e30 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.385+0000 7fe94e7fc640 1 -- 172.21.15.135:0/1389301955 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9480019f0 con 0x7fe950158e30 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.386+0000 7fe956124640 1 -- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158e30 msgr2=0x7fe95006f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.386+0000 7fe956124640 1 --2- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158e30 0x7fe95006f610 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe94400a940 tx=0x7fe94402cc60 comp rx=0 tx=0).stop 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.386+0000 7fe956124640 1 -- 172.21.15.135:0/1389301955 shutdown_connections 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.386+0000 7fe956124640 1 --2- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158e30 0x7fe95006f610 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.386+0000 7fe956124640 1 --2- 172.21.15.135:0/1389301955 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158a50 0x7fe95015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.386 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.386+0000 7fe956124640 1 -- 172.21.15.135:0/1389301955 >> 172.21.15.135:0/1389301955 conn(0x7fe9500d0520 msgr2=0x7fe9500d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:01.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.386+0000 7fe956124640 1 -- 172.21.15.135:0/1389301955 shutdown_connections 2024-04-17T06:11:01.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.386+0000 7fe956124640 1 -- 172.21.15.135:0/1389301955 wait complete. 2024-04-17T06:11:01.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.387+0000 7fe956124640 1 Processor -- start 2024-04-17T06:11:01.387 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.387+0000 7fe956124640 1 -- start start 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe956124640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9500b3590 0x7fe950159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe956124640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158a50 0x7fe9500ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe956124640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158e30 0x7fe950155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe956124640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe95015c3e0 con 0x7fe9500b3590 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe956124640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7fe95015f8a0 con 0x7fe950158a50 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe956124640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7fe95015c560 con 0x7fe950158e30 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe94f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9500b3590 0x7fe950159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe94f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9500b3590 0x7fe950159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53232/0 (socket says 172.21.15.135:53232) 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe94f7fe640 1 -- 172.21.15.135:0/2147236102 learned_addr learned my addr 172.21.15.135:0/2147236102 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.388+0000 7fe94effd640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158a50 0x7fe9500ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.390 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe94ffff640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158e30 0x7fe950155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe94f7fe640 1 -- 172.21.15.135:0/2147236102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158e30 msgr2=0x7fe950155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe94f7fe640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158e30 0x7fe950155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe94f7fe640 1 -- 172.21.15.135:0/2147236102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158a50 msgr2=0x7fe9500ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe94f7fe640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158a50 0x7fe9500ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe94f7fe640 1 -- 172.21.15.135:0/2147236102 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9500acb70 con 0x7fe9500b3590 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe94f7fe640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9500b3590 0x7fe950159480 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe94000ec10 tx=0x7fe94000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe937fff640 1 -- 172.21.15.135:0/2147236102 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940017070 con 0x7fe9500b3590 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe937fff640 1 -- 172.21.15.135:0/2147236102 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94000cc30 con 0x7fe9500b3590 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9501f28e0 con 0x7fe9500b3590 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe937fff640 1 -- 172.21.15.135:0/2147236102 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940013470 con 0x7fe9500b3590 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.389+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9501f2ea0 con 0x7fe9500b3590 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.390+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe950074940 con 0x7fe9500b3590 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.390+0000 7fe937fff640 1 -- 172.21.15.135:0/2147236102 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe94001a070 con 0x7fe9500b3590 2024-04-17T06:11:01.391 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.391+0000 7fe937fff640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe93c06c8f0 0x7fe93c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.391+0000 7fe94effd640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe93c06c8f0 0x7fe93c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.391+0000 7fe937fff640 1 -- 172.21.15.135:0/2147236102 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7fe94008bf80 con 0x7fe9500b3590 2024-04-17T06:11:01.392 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.391+0000 7fe94effd640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe93c06c8f0 0x7fe93c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe944037000 tx=0x7fe944008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.393 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.395+0000 7fe937fff640 1 -- 172.21.15.135:0/2147236102 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe94005a290 con 0x7fe9500b3590 2024-04-17T06:11:01.431 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.434 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.438 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.496 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.497 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.550 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.553 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.556 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.566 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.568+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe9500cd010 con 0x7fe9500b3590 2024-04-17T06:11:01.567 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.569+0000 7fe937fff640 1 -- 172.21.15.135:0/2147236102 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7fe940059b90 con 0x7fe9500b3590 2024-04-17T06:11:01.567 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 32 2024-04-17T06:11:01.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe93c06c8f0 msgr2=0x7fe93c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe93c06c8f0 0x7fe93c06edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe944037000 tx=0x7fe944008040 comp rx=0 tx=0).stop 2024-04-17T06:11:01.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9500b3590 msgr2=0x7fe950159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9500b3590 0x7fe950159480 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe94000ec10 tx=0x7fe94000c520 comp rx=0 tx=0).stop 2024-04-17T06:11:01.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 shutdown_connections 2024-04-17T06:11:01.569 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe950158e30 0x7fe950155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe950158a50 0x7fe9500ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7fe93c06c8f0 0x7fe93c06edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 --2- 172.21.15.135:0/2147236102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9500b3590 0x7fe950159480 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 >> 172.21.15.135:0/2147236102 conn(0x7fe9500d0520 msgr2=0x7fe950156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:01.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 shutdown_connections 2024-04-17T06:11:01.570 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.571+0000 7fe956124640 1 -- 172.21.15.135:0/2147236102 wait complete. 2024-04-17T06:11:01.580 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-17T06:11:01.615 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.615 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.658 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.669 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.671 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.675 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.687+0000 7f8c5c532640 1 Processor -- start 2024-04-17T06:11:01.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.688+0000 7f8c5c532640 1 -- start start 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.688+0000 7f8c5c532640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c5415bb00 0x7f8c540b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.688+0000 7f8c5c532640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540b3590 0x7f8c540af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.688+0000 7f8c5c532640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c540afc00 0x7f8c54071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.688+0000 7f8c5c532640 1 -- --> v2:172.21.15.135:3300/0 -- mon_getmap magic: 0 -- 0x7f8c540b0080 con 0x7f8c540afc00 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.688+0000 7f8c5c532640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 -- 0x7f8c540b4660 con 0x7f8c5415bb00 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.688+0000 7f8c5c532640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8c540b3a50 con 0x7f8c540b3590 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5aacf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c540afc00 0x7f8c54071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5aacf640 1 --2- >> v2:172.21.15.135:3300/0 conn(0x7f8c540afc00 0x7f8c54071ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53238/0 (socket says 172.21.15.135:53238) 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5aacf640 1 -- 172.21.15.135:0/2181521481 learned_addr learned my addr 172.21.15.135:0/2181521481 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c59acd640 1 --2- 172.21.15.135:0/2181521481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540b3590 0x7f8c540af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.688 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5aacf640 1 -- 172.21.15.135:0/2181521481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540b3590 msgr2=0x7f8c540af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5a2ce640 1 --2- 172.21.15.135:0/2181521481 >> v2:172.21.15.190:3300/0 conn(0x7f8c5415bb00 0x7f8c540b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5aacf640 1 --2- 172.21.15.135:0/2181521481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540b3590 0x7f8c540af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5aacf640 1 -- 172.21.15.135:0/2181521481 >> v2:172.21.15.190:3300/0 conn(0x7f8c5415bb00 msgr2=0x7f8c540b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5aacf640 1 --2- 172.21.15.135:0/2181521481 >> v2:172.21.15.190:3300/0 conn(0x7f8c5415bb00 0x7f8c540b5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5aacf640 1 -- 172.21.15.135:0/2181521481 --> v2:172.21.15.135:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c540b3d30 con 0x7f8c540afc00 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.689+0000 7f8c5a2ce640 1 --2- 172.21.15.135:0/2181521481 >> v2:172.21.15.190:3300/0 conn(0x7f8c5415bb00 0x7f8c540b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c5aacf640 1 --2- 172.21.15.135:0/2181521481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540afc00 0x7f8c54071ec0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8c5406dc60 tx=0x7f8c4402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672331a25293f0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c592cc640 1 -- 172.21.15.135:0/2181521481 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4403b070 con 0x7f8c540afc00 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c592cc640 1 -- 172.21.15.135:0/2181521481 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c44036440 con 0x7f8c540afc00 2024-04-17T06:11:01.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c592cc640 1 -- 172.21.15.135:0/2181521481 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44036810 con 0x7f8c540afc00 2024-04-17T06:11:01.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c5c532640 1 -- 172.21.15.135:0/2181521481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540afc00 msgr2=0x7f8c54071ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c5c532640 1 --2- 172.21.15.135:0/2181521481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540afc00 0x7f8c54071ec0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8c5406dc60 tx=0x7f8c4402e0e0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c5c532640 1 -- 172.21.15.135:0/2181521481 shutdown_connections 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c5c532640 1 --2- 172.21.15.135:0/2181521481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540afc00 0x7f8c54071ec0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c5c532640 1 --2- 172.21.15.135:0/2181521481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540b3590 0x7f8c540af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c5c532640 1 --2- 172.21.15.135:0/2181521481 >> v2:172.21.15.190:3300/0 conn(0x7f8c5415bb00 0x7f8c540b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.690+0000 7f8c5c532640 1 -- 172.21.15.135:0/2181521481 >> 172.21.15.135:0/2181521481 conn(0x7f8c540d0520 msgr2=0x7f8c540d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.691+0000 7f8c5c532640 1 -- 172.21.15.135:0/2181521481 shutdown_connections 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.691+0000 7f8c5c532640 1 -- 172.21.15.135:0/2181521481 wait complete. 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.692+0000 7f8c5c532640 1 Processor -- start 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.692+0000 7f8c5c532640 1 -- start start 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.692+0000 7f8c5c532640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540afc00 0x7f8c54151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.692+0000 7f8c5c532640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540b3590 0x7f8c54159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.692+0000 7f8c5c532640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c5415bb00 0x7f8c54146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.693 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.692+0000 7f8c5c532640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c540b5690 con 0x7f8c540b3590 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.692+0000 7f8c5c532640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f8c540b42f0 con 0x7f8c5415bb00 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.692+0000 7f8c5c532640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f8c540b5810 con 0x7f8c540afc00 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540b3590 0x7f8c54159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540b3590 0x7f8c54159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53254/0 (socket says 172.21.15.135:53254) 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 -- 172.21.15.135:0/3698266491 learned_addr learned my addr 172.21.15.135:0/3698266491 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c5aacf640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c5415bb00 0x7f8c54146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 -- 172.21.15.135:0/3698266491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540afc00 msgr2=0x7f8c54151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c5a2ce640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540afc00 0x7f8c54151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540afc00 0x7f8c54151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 -- 172.21.15.135:0/3698266491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c5415bb00 msgr2=0x7f8c54146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c5415bb00 0x7f8c54146aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 -- 172.21.15.135:0/3698266491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c54152150 con 0x7f8c540b3590 2024-04-17T06:11:01.694 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c59acd640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540b3590 0x7f8c54159e40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800cd20 tx=0x7f8c4800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c52ffd640 1 -- 172.21.15.135:0/3698266491 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48003b90 con 0x7f8c540b3590 2024-04-17T06:11:01.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c52ffd640 1 -- 172.21.15.135:0/3698266491 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c48003d30 con 0x7f8c540b3590 2024-04-17T06:11:01.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c54152a80 con 0x7f8c540b3590 2024-04-17T06:11:01.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c52ffd640 1 -- 172.21.15.135:0/3698266491 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48004270 con 0x7f8c540b3590 2024-04-17T06:11:01.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.693+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541f13e0 con 0x7f8c540b3590 2024-04-17T06:11:01.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.694+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c54074940 con 0x7f8c540b3590 2024-04-17T06:11:01.695 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.695+0000 7f8c52ffd640 1 -- 172.21.15.135:0/3698266491 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c48004410 con 0x7f8c540b3590 2024-04-17T06:11:01.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.695+0000 7f8c52ffd640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f8c4006c8f0 0x7f8c4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.695+0000 7f8c5a2ce640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f8c4006c8f0 0x7f8c4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.695+0000 7f8c52ffd640 1 -- 172.21.15.135:0/3698266491 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f8c4808c690 con 0x7f8c540b3590 2024-04-17T06:11:01.696 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.696+0000 7f8c5a2ce640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f8c4006c8f0 0x7f8c4006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8c54157460 tx=0x7f8c4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.697 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.700+0000 7f8c52ffd640 1 -- 172.21.15.135:0/3698266491 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8c4801d340 con 0x7f8c540b3590 2024-04-17T06:11:01.724 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.724 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.741 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.741 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.741 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.778 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.807 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.808 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.808 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.810 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.845 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.873 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.873 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.873 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.874 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.877+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8c540b4b10 con 0x7f8c540b3590 2024-04-17T06:11:01.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.877+0000 7f8c52ffd640 1 -- 172.21.15.135:0/3698266491 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f8c4805a9a0 con 0x7f8c540b3590 2024-04-17T06:11:01.875 INFO:teuthology.orchestra.run.smithi135.stdout:pg_num: 64 2024-04-17T06:11:01.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f8c4006c8f0 msgr2=0x7f8c4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f8c4006c8f0 0x7f8c4006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8c54157460 tx=0x7f8c4c008040 comp rx=0 tx=0).stop 2024-04-17T06:11:01.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540b3590 msgr2=0x7f8c54159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540b3590 0x7f8c54159e40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800cd20 tx=0x7f8c4800a660 comp rx=0 tx=0).stop 2024-04-17T06:11:01.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 shutdown_connections 2024-04-17T06:11:01.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c5415bb00 0x7f8c54146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f8c4006c8f0 0x7f8c4006edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.878 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c540b3590 0x7f8c54159e40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 --2- 172.21.15.135:0/3698266491 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c540afc00 0x7f8c54151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.879+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 >> 172.21.15.135:0/3698266491 conn(0x7f8c540d0520 msgr2=0x7f8c540b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:01.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.880+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 shutdown_connections 2024-04-17T06:11:01.879 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.880+0000 7f8c5c532640 1 -- 172.21.15.135:0/3698266491 wait complete. 2024-04-17T06:11:01.889 DEBUG:teuthology.orchestra.run.smithi135:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-17T06:11:01.900 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.939 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.940 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.940 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.942 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.962 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:01.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.994+0000 7f9a759c8640 1 Processor -- start 2024-04-17T06:11:01.992 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.994+0000 7f9a759c8640 1 -- start start 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a759c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a7015f800 0x7f9a70070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a759c8640 1 -- --> v1:172.21.15.135:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a7006bb30 con 0x7f9a70070c90 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a759c8640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a7006be10 con 0x7f9a700d0f20 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a759c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7006c8f0 con 0x7f9a7015f800 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a667fc640 1 --1- >> v1:172.21.15.135:6789/0 conn(0x7f9a70070c90 0x7f9a700d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.135:6789/0 says I am v1:172.21.15.135:39126/0 (socket says 172.21.15.135:39126) 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a667fc640 1 -- 172.21.15.135:0/3401410579 learned_addr learned my addr 172.21.15.135:0/3401410579 (peer_addr_for_me v1:172.21.15.135:0/0) 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 <== mon.0 v1:172.21.15.135:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502923127 0 0) 0x7f9a7006bb30 con 0x7f9a70070c90 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a6effd640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a7015f800 0x7f9a70070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.995+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 --> v1:172.21.15.135:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a5c003520 con 0x7f9a70070c90 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 <== mon.0 v1:172.21.15.135:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1441629683 0 0) 0x7f9a5c003520 con 0x7f9a70070c90 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a7015f800 msgr2=0x7f9a70070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:01.996 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a7015f800 0x7f9a70070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 >> v1:172.21.15.190:6789/0 conn(0x7f9a700d0f20 legacy=0x7f9a700cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 --> v1:172.21.15.135:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7006c0f0 con 0x7f9a70070c90 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 <== mon.0 v1:172.21.15.135:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 270383572 0 0) 0x7f9a58001690 con 0x7f9a70070c90 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 >> v1:172.21.15.135:6789/0 conn(0x7f9a70070c90 legacy=0x7f9a700d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a5c0039e0 0x7f9a5c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a7015f800 0x7f9a5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7006c8f0 con 0x7f9a5c004d40 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c00bb00 con 0x7f9a5c0039e0 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5c00bd00 con 0x7f9a7015f800 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6f7fe640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6effd640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a5c0039e0 0x7f9a5c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a667fc640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a7015f800 0x7f9a5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6f7fe640 1 -- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a7015f800 msgr2=0x7f9a5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6f7fe640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a7015f800 0x7f9a5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6f7fe640 1 -- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a5c0039e0 msgr2=0x7f9a5c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6f7fe640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a5c0039e0 0x7f9a5c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.996+0000 7f9a6f7fe640 1 -- 172.21.15.135:0/3401410579 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c003520 con 0x7f9a5c004d40 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.997+0000 7f9a6f7fe640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a5c0051c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9a700c8600 tx=0x7f9a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28906d52a1b17e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.997+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68001a70 con 0x7f9a5c004d40 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.997+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a68003b70 con 0x7f9a5c004d40 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.997+0000 7f9a6e7fc640 1 -- 172.21.15.135:0/3401410579 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680050d0 con 0x7f9a5c004d40 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.998+0000 7f9a759c8640 1 -- 172.21.15.135:0/3401410579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 msgr2=0x7f9a5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.998+0000 7f9a759c8640 1 --2- 172.21.15.135:0/3401410579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a5c0051c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9a700c8600 tx=0x7f9a5c005830 comp rx=0 tx=0).stop 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.998+0000 7f9a6f7fe640 1 -- 172.21.15.135:0/3401410579 reap_dead start 2024-04-17T06:11:01.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.998+0000 7f9a759c8640 1 -- 172.21.15.135:0/3401410579 shutdown_connections 2024-04-17T06:11:01.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.998+0000 7f9a759c8640 1 -- 172.21.15.135:0/3401410579 >> 172.21.15.135:0/3401410579 conn(0x7f9a7009a9f0 msgr2=0x7f9a700b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:01.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.998+0000 7f9a759c8640 1 -- 172.21.15.135:0/3401410579 shutdown_connections 2024-04-17T06:11:01.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.998+0000 7f9a759c8640 1 -- 172.21.15.135:0/3401410579 wait complete. 2024-04-17T06:11:02.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.999+0000 7f9a759c8640 1 Processor -- start 2024-04-17T06:11:02.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:01.999+0000 7f9a759c8640 1 -- start start 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a759c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a7015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a759c8640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a701f4a60 0x7f9a7015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a759c8640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a7015e060 0x7f9a700b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a759c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7015fe30 con 0x7f9a5c004d40 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a759c8640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7006c460 con 0x7f9a7015e060 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a759c8640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7015b000 con 0x7f9a701f4a60 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a7015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a7015bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:53278/0 (socket says 172.21.15.135:53278) 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 -- 172.21.15.135:0/276616699 learned_addr learned my addr 172.21.15.135:0/276616699 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6f7fe640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a7015e060 0x7f9a700b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 -- 172.21.15.135:0/276616699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a701f4a60 msgr2=0x7f9a7015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a701f4a60 0x7f9a7015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 -- 172.21.15.135:0/276616699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a7015e060 msgr2=0x7f9a700b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:02.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a7015e060 0x7f9a700b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 -- 172.21.15.135:0/276616699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7015b2e0 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6effd640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a7015bdc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9a600119a0 tx=0x7f9a60011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6cff9640 1 -- 172.21.15.135:0/276616699 <== mon.0 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000faf0 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6cff9640 1 -- 172.21.15.135:0/276616699 <== mon.0 v2:172.21.15.135:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6000fc90 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7015b5c0 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.000+0000 7f9a6cff9640 1 -- 172.21.15.135:0/276616699 <== mon.0 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60014440 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.001+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7015a460 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.002+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70074940 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.002+0000 7f9a6cff9640 1 -- 172.21.15.135:0/276616699 <== mon.0 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a60012040 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.002+0000 7f9a6cff9640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a5006c8f0 0x7f9a5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.002+0000 7f9a6cff9640 1 -- 172.21.15.135:0/276616699 <== mon.0 v2:172.21.15.135:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6130+0+0 (secure 0 0 0) 0x7f9a6008c3e0 con 0x7f9a5c004d40 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.002+0000 7f9a667fc640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a5006c8f0 0x7f9a5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-17T06:11:02.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.003+0000 7f9a667fc640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a5006c8f0 0x7f9a5006edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a58001cf0 tx=0x7f9a58001b80 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-17T06:11:02.005 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.007+0000 7f9a6cff9640 1 -- 172.21.15.135:0/276616699 <== mon.0 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a60019030 con 0x7f9a5c004d40 2024-04-17T06:11:02.006 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.007 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.007 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.009 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.019 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.073 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.080 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.127 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.127 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.127 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.136 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.192 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.193 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.196+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9a700ceeb0 con 0x7f9a5c004d40 2024-04-17T06:11:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.196+0000 7f9a6cff9640 1 -- 172.21.15.135:0/276616699 <== mon.0 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) ==== 80+0+0 (secure 0 0 0) 0x7f9a6005a6f0 con 0x7f9a5c004d40 2024-04-17T06:11:02.194 INFO:teuthology.orchestra.run.smithi135.stderr:listed 0 entries 2024-04-17T06:11:02.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a5006c8f0 msgr2=0x7f9a5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:02.196 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a5006c8f0 0x7f9a5006edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a58001cf0 tx=0x7f9a58001b80 comp rx=0 tx=0).stop 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 msgr2=0x7f9a7015bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a7015bdc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9a600119a0 tx=0x7f9a60011e60 comp rx=0 tx=0).stop 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 shutdown_connections 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a7015e060 0x7f9a700b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a701f4a60 0x7f9a7015db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:6832/3190814219,v1:172.21.15.135:6833/3190814219] conn(0x7f9a5006c8f0 0x7f9a5006edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 --2- 172.21.15.135:0/276616699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a5c004d40 0x7f9a7015bdc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 >> 172.21.15.135:0/276616699 conn(0x7f9a7009a9f0 msgr2=0x7f9a700afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 shutdown_connections 2024-04-17T06:11:02.197 INFO:teuthology.orchestra.run.smithi135.stderr:2024-04-17T06:11:02.198+0000 7f9a759c8640 1 -- 172.21.15.135:0/276616699 wait complete. 2024-04-17T06:11:02.199 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.207 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-17T06:11:02.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:11:02.207 DEBUG:teuthology.orchestra.run.smithi135:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-17T06:11:02.248 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.248 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.249 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.251 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.311 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.318 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.364 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.366 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.366 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.366 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.430 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.432 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.481 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.484 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.486 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi135.front.sepia.ceph.com 2024-04-17T06:11:02.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-17T06:11:02.544 DEBUG:teuthology.orchestra.run.smithi135:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-17T06:11:02.548 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.550 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.560 INFO:tasks.cephfs.mount:Terminating background process 2024-04-17T06:11:02.560 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_0", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:02.562 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:02 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:02.601 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.603 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.607 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.667 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.668 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.719 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.720 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.725 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.784 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.786 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.838 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.842 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.843 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.904 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.904 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.958 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.960 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:02.968 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.023 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.025 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.077 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.079 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.086 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.144 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.144 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.196 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.196 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.204 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.260 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.263 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.313 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.316 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.323 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.379 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.380 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.433 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.436 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.441 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.498 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.499 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.552 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.555 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.558 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.617 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.618 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.660 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.665 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:03.670 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.673 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_2", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:03.673 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.674 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:03 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:03.676 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.726 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.727 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.743 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.743 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.744 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.780 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.808 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.810 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.810 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.812 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.847 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.875 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.875 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.876 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.876 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.902 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.907 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:03.915 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_4", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:03.942 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.942 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.942 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.943 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:03 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:03.944 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:03.963 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.008 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.009 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.009 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.011 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.021 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.075 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.082 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.129 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.129 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.130 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.138 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.143 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:04.152 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_6", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:04.193 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:04 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:04.194 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.201 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.248 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.248 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.251 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.254 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.313 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.320 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.366 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.368 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.368 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.368 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.373 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:04.382 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_8", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:04.423 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:04 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:04.432 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.433 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.482 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.484 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.486 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.551 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.552 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.603 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.605 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.609 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.614 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:04.623 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_10", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:04.664 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:04 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:04.669 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.670 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.720 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.722 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.727 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.786 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.787 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.840 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.844 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.845 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.850 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:04.858 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_12", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:04.900 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:04 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:04.905 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.906 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.961 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.962 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:04.970 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.025 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.027 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.079 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.081 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.088 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.092 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:05.101 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_14", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:05.142 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:05.146 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.146 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.198 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.198 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.206 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.262 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.265 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.316 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.318 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.325 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.330 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:05.339 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_16", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:05.380 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:05.381 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.382 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.435 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.437 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.443 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.500 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.501 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.554 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.557 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.560 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.565 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:05.574 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_18", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:05.615 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:05.619 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.619 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.672 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.676 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.679 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.728 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.729 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.744 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.745 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.745 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.749 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:05.758 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_20", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:05.782 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.782 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:05.810 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.812 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.812 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.814 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.849 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.876 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.877 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.878 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.878 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.944 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.944 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.944 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.946 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.965 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:05.969 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:05.978 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_22", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:06.010 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.010 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:06.011 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.011 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.012 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.023 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.077 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.084 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.131 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.131 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.131 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.196 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.203 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.207 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:06.215 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_24", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:06.250 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.251 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.251 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:06.253 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.256 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.315 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.322 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.368 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.370 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.370 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.435 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.436 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.440 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:06.450 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_26", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:06.485 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.485 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:06.486 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.488 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.552 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.554 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.605 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.607 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.672 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.672 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.676 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:06.684 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_28", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:06.721 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.721 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:06.723 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.729 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.788 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.788 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.842 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.846 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.908 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.908 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.913 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:06.922 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_30", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:06.963 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.963 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:06 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:06.964 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:06.972 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.027 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.029 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.079 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.083 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.148 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.148 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.152 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:07.160 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_32", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:07.200 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.200 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.201 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:07.208 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.264 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.267 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.318 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.321 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.383 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.384 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.387 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:07.395 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_34", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:07.437 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.437 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:07.440 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.443 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.502 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.504 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.556 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.559 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.621 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.622 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.626 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:07.634 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_36", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:07.674 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.674 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:07.678 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.681 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.730 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.730 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.746 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.748 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.783 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.812 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.814 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.814 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.816 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.816 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:07.824 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_38", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:07.851 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.852 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:07.878 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.879 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.880 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.880 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.946 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.946 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.946 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.948 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:07.952 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:07.960 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_40", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:08.002 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:08.011 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.013 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.014 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.014 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.025 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.080 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.086 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.133 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.133 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.134 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.137 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:08.146 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_42", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:08.188 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:08.196 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.252 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.252 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.255 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.258 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.317 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.324 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.370 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.372 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.372 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.375 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:08.384 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_44", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:08.426 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:08.437 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.486 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.489 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.490 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.554 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.556 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.607 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.609 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.612 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:08.621 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_46", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:08.662 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:08.673 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.722 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.726 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.731 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.790 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.790 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.844 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.848 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.849 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:08.858 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_48", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:08.901 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:08 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:08.909 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.964 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.966 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:08.974 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.029 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.031 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.081 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.084 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.086 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:09.096 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_50", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:09.137 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:09.149 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.202 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.203 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.210 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.265 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.269 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.320 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.322 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.325 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:09.333 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_52", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:09.374 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:09.385 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.439 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.442 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.446 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.504 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.506 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.558 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.560 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.563 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:09.571 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_54", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:09.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:09.624 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.676 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.680 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.683 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.732 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.733 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.748 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.750 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.754 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:09.763 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_56", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:09.786 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.786 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:09.816 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.816 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.818 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.853 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.880 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.880 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.881 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.882 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.885 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:09.893 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_58", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:09.934 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:09 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:09.947 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.947 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:09.948 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.014 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.015 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.015 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.016 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.019 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:10.027 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_60", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:10.028 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.069 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:10.081 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.088 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.135 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.136 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.199 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.254 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.255 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.257 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.259 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:10.260 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.268 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_62", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:10.309 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:10.319 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.326 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.374 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.374 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.439 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.488 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.490 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.491 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.495 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:10.504 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_64", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:10.546 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:10.557 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.558 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.611 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.676 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.725 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.728 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.733 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:10.733 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.741 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_66", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:10.784 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:10.792 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.792 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.850 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.911 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.967 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.968 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.973 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:10.976 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:10.981 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_68", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:11.023 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:11.031 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.033 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.087 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.151 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.204 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.205 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.209 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:11.212 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.219 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_70", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:11.260 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:11.266 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.271 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.325 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.388 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.441 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.444 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.448 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.448 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:11.457 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_72", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:11.498 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:11.505 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.508 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.563 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.626 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.678 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.682 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.684 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.687 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:11.695 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_74", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:11.734 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.734 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.735 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:11.750 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.788 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.818 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.818 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.820 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.822 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:11.830 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_76", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:11.854 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.855 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:11.883 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.884 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.884 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.949 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.949 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.949 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:11.953 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:11.962 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_78", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:12.004 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:12.017 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.017 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.019 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.029 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.083 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.089 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:12.090 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.097 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_80", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:12.137 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.138 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.138 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:12.201 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.256 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.260 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.262 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.321 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.326 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:12.327 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.334 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_82", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:12.376 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.376 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.376 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:12.441 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.490 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.492 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.558 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.559 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.563 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:12.570 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_84", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:12.613 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:12.613 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.678 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.727 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.735 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.794 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.794 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.798 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:12.806 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_86", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:12.847 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:12 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:12.851 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.913 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.969 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:12.978 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.033 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.035 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.037 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:13.046 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_88", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:13.087 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:13 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:13.089 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.153 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.206 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.214 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.268 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.273 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:13.273 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.280 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_90", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:13.322 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:13 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:13.327 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.390 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.443 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.450 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.508 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.510 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.514 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-17T06:11:13.522 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nwith open("/home/ubuntu/cephtest/mnt.1/test_dir/test_file_92", \'r\') as f:\n while True:\n print("open_background: keeping file open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\n'], check_status=True, hostname='smithi190') 2024-04-17T06:11:13.564 INFO:teuthology.orchestra.run.smithi190.stderr:2024-04-17T06:11:13 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-17T06:11:13.565 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.628 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.680 INFO:teuthology.orchestra.run.smithi190.stderr:open_background: keeping file open 2024-04-17T06:11:13.682 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-04-17T06:11:13.682 DEBUG:teuthology.task.console_log:Killing console logger for smithi135 2024-04-17T06:11:13.683 DEBUG:teuthology.task.console_log:Killing console logger for smithi190 2024-04-17T06:11:13.683 DEBUG:teuthology.exit:Finished running handlers